Conformal Prediction Intervals for Remaining Useful Lifetime Estimation
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Jul 24, 2023
Alireza Javanmardi
Eyke Hüllermeier
https://orcid.org/0000-0002-9944-4108
Abstract
The main objective of Prognostics and Health Management is to estimate the Remaining Useful Lifetime (RUL), namely, the time that a system or a piece of equipment is still in working order before starting to function incorrectly. In recent years, numerous machine learning algorithms have been proposed for RUL estimation, mainly focusing on providing more accurate RUL predictions. However, there are many sources of uncertainty in the problem, such as inherent randomness of systems failure, lack of knowledge regarding their future states, and inaccuracy of the underlying predictive models, making it infeasible to predict the RULs precisely. Hence, it is of utmost importance to quantify the uncertainty alongside the RUL predictions. In this work, we investigate the conformal prediction (CP) framework that represents uncertainty by predicting sets of possible values for the target variable (intervals in the case of RUL) instead of making point predictions. Under very mild technical assumptions, CP formally guarantees that the actual value (true RUL) is covered by the predicted set with a degree of certainty that can be prespecified. We study three CP algorithms to conformalize any single-point RUL predictor and turn it into a valid interval predictor. Finally, we conformalize two single-point RUL predictors, deep convolutional neural networks and gradient boosting, and illustrate their performance on the C-MAPSS datasets.
##plugins.themes.bootstrap3.article.details##
Keywords
Remaining useful lifetime estimation, Machine learning, Uncertainty quantification, Conformal prediction
References
An, D., Kim, N. H., & Choi, J. H. (2015). Practical options for selecting data-driven or physics-based prognostics algorithms with reviews (Vol. 133). Retrieved from http://dx.doi.org/10.1016/j.ress.2014.09.014 doi: 10.1016/j.ress.2014.09.014
Angelopoulos, A. N., & Bates, S. (2021). A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. arXiv. Retrieved from http://arxiv.org/abs/2107.07511
Babu, G. S., Zhao, P., & Li, X. L. (2016). Deep convolutional neural network based regression approach for estimation of remaining useful life. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 9642, pp. 214–228). Springer Verlag. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-32025-0_14 doi: 10.1007/978-3-319-32025-0_14
Barber, R. F., Candes, E. J., Ramdas, A., & Tibshirani, R. J. (2022). Conformal prediction beyond exchangeability. arXiv. Retrieved from http://arxiv.org/abs/2202.13415
Benkedjouh, T., Medjaher, K., Zerhouni, N., & Rechak, S. (2013). Remaining useful life estimation based on nonlinear feature reduction and support vector regression. Engineering Applications of Artificial Intelligence, 26(7), 1751–1760. Retrieved from http://dx.doi.org/10.1016/j.engappai.2013.02.006 doi: 10.1016/j.engappai.2013.02.006
Benker, M., Furtner, L., Semm, T., & Zaeh, M. F. (2021, oct). Utilizing uncertainty information in remaining useful life estimation via Bayesian neural networks and Hamiltonian Monte Carlo. Journal of Manufacturing Systems, 61, 799–807. doi: 10.1016/j.jmsy.2020.11.005
Biggio, L.,Wieland, A., Chao, M. A., Kastanis, I., & Fink, O. (2021). Uncertainty-aware Remaining Useful Life predictor. arXiv. Retrieved from http://arxiv.org/abs/2104.03613
Chen, J., Jing, H., Chang, Y., & Liu, Q. (2019). Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process. Reliability Engineering and System Safety, 185(January), 372–382. doi: 10.1016/j.ress.2019.01.006
Dewolf, N., Baets, B. D., & Waegeman, W. (2022). Valid prediction intervals for regression problems. Artificial Intelligence Review. Retrieved from https://doi.org/10.1007/s10462-022-10178-5 doi: 10.1007/s10462-022-10178-5
Elsheikh, A., Yacout, S., & Ouali, M. S. (2019). Bidirectional handshaking LSTM for remaining useful life prediction. Neurocomputing, 323, 148–156. Retrieved from https://doi.org/10.1016/j.neucom.2018.09.076 doi: 10.1016/j.neucom.2018.09.076
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, PHM 2008. doi: 10.1109/PHM.2008.4711422
Hüllermeier, E., &Waegeman,W. (2021). Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3), 457–506. Retrieved from https://doi.org/10.1007/s10994-021-05946-3 doi: 10.1007/s10994-021-05946-3
Jardine, A. K., Lin, D., & Banjevic, D. (2006, oct). A review on machinery diagnostics and prognostics implementing condition-based maintenance (Vol. 20) (No. 7). doi: 10.1016/j.ymssp.2005.09.012
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-Free Predictive Inference for Regression. Journal of the American Statistical Association, 113(523), 1094–1111. Retrieved from https://doi.org/10.1080/01621459.2017.1307116 doi: 10.1080/01621459.2017.1307116
Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018, may). Machinery health prognostics: A systematic review from data acquisition to RUL prediction (Vol. 104). Academic Press. doi: 10.1016/j.ymssp.2017.11.016
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering and System Safety, 172, 1–11. Retrieved from http://www.elsevier.com/open-access/userlicense/1.0/ doi: 10.1016/j.ress.2017.11.021
Liao, Y., Zhang, L., & Liu, C. (2018, aug). Uncertainty Prediction of Remaining Useful Life Using Long Short-Term Memory Network Based on Bootstrap Method. In 2018 ieee international conference on prognostics and health management, icphm 2018. Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/ICPHM.2018.8448804
Liu, C., Zhang, L., Liao, Y., Wu, C., & Peng, G. (2019). Multiple Sensors Based Prognostics with Prediction Interval Optimization via Echo State Gaussian Process. IEEE Access, 7, 112397–112409. doi: 10.1109/ACCESS.2019.2925634
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on Bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048. doi: 10.1007/s10845-014-0933-4
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830. Retrieved from http://scikit-learn.sourceforge.net.
Peng, W., Ye, Z. S., & Chen, N. (2020). Bayesian Deep-Learning-Based Health Prognostics Toward Prognostics Uncertainty. IEEE Transactions on Industrial Electronics, 67(3), 2283–2293. doi: 10.1109/TIE.2019.2907440
Rigamonti, M., Baraldi, P., Zio, E., Roychoudhury, I., Goebel, K., & Poll, S. (2018). Ensemble of optimized echo state networks for remaining useful life prediction. Neurocomputing, 281, 121–138. Retrieved from https://doi.org/10.1016/j.neucom.2017.11.062 doi: 10.1016/j.neucom.2017.11.062
Romano, Y., Patterson, E., & Cand`es, E. J. (2019). Conformalized quantile regression. In Advances in neural information processing systems (Vol. 32). Retrieved from https://github.com/yromano/cqr.
Sankararaman, S., & Goebel, K. (2015). Uncertainty in prognostics and systems health management. International Journal of Prognostics and Health Management, 6, 1–14. doi: 10.36001/ijphm.2015.v6i4.2319
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management, phm 2008. doi: 10.1109/PHM.2008.4711414
Tornede, T., Tornede, A., Wever, M., & H¨ullermeier, E. (2021). Coevolution of remaining useful lifetime estimation pipelines for automated predictive maintenance. In Gecco 2021 - proceedings of the 2021 genetic and evolutionary computation conference (pp. 368–376). Retrieved from https://doi.org/10.1145/3449639.3459395 doi: 10.1145/3449639.3459395
Tornede, T., Tornede, A., Wever, M., Mohr, F., & Hüllermeier, E. (2020). AutoML for Predictive Maintenance: One Tool to RUL Them All. In Communications in computer and information science (Vol. 1325, pp. 106–118). Retrieved from https://doi.org/10.1007/978-3-030-66770-2_8 doi: 10.1007/978-3-030-66770-2_8
Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Springer-Verlag. doi: 10.1007/b106715
Wu, Q., Ding, K., & Huang, B. (2020, oct). Approach for fault prognosis using recurrent neural network. Journal of Intelligent Manufacturing, 31(7), 1621–1633. Retrieved from https://link.springer.com/article/10.1007/s10845-018-1428-5 doi: 10.1007/s10845-018-1428-5
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275, 167–179. Retrieved from http://dx.doi.org/10.1016/j.neucom.2017.05.063 doi: 10.1016/j.neucom.2017.05.063
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017, oct). Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2306–2318. doi: 10.1109/TNNLS.2016.2582798
Zhao, Z., Wu, J., Wong, D., Sun, C., & Yan, R. (2020). Probabilistic Remaining Useful Life Prediction Based on Deep Convolutional Neural Network. SSRN Electronic Journal. Retrieved from https://github.com/ZhaoZhibin/Probabilistic RUL Prediction. doi: 10.2139/ssrn.3717738
Angelopoulos, A. N., & Bates, S. (2021). A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. arXiv. Retrieved from http://arxiv.org/abs/2107.07511
Babu, G. S., Zhao, P., & Li, X. L. (2016). Deep convolutional neural network based regression approach for estimation of remaining useful life. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 9642, pp. 214–228). Springer Verlag. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-32025-0_14 doi: 10.1007/978-3-319-32025-0_14
Barber, R. F., Candes, E. J., Ramdas, A., & Tibshirani, R. J. (2022). Conformal prediction beyond exchangeability. arXiv. Retrieved from http://arxiv.org/abs/2202.13415
Benkedjouh, T., Medjaher, K., Zerhouni, N., & Rechak, S. (2013). Remaining useful life estimation based on nonlinear feature reduction and support vector regression. Engineering Applications of Artificial Intelligence, 26(7), 1751–1760. Retrieved from http://dx.doi.org/10.1016/j.engappai.2013.02.006 doi: 10.1016/j.engappai.2013.02.006
Benker, M., Furtner, L., Semm, T., & Zaeh, M. F. (2021, oct). Utilizing uncertainty information in remaining useful life estimation via Bayesian neural networks and Hamiltonian Monte Carlo. Journal of Manufacturing Systems, 61, 799–807. doi: 10.1016/j.jmsy.2020.11.005
Biggio, L.,Wieland, A., Chao, M. A., Kastanis, I., & Fink, O. (2021). Uncertainty-aware Remaining Useful Life predictor. arXiv. Retrieved from http://arxiv.org/abs/2104.03613
Chen, J., Jing, H., Chang, Y., & Liu, Q. (2019). Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process. Reliability Engineering and System Safety, 185(January), 372–382. doi: 10.1016/j.ress.2019.01.006
Dewolf, N., Baets, B. D., & Waegeman, W. (2022). Valid prediction intervals for regression problems. Artificial Intelligence Review. Retrieved from https://doi.org/10.1007/s10462-022-10178-5 doi: 10.1007/s10462-022-10178-5
Elsheikh, A., Yacout, S., & Ouali, M. S. (2019). Bidirectional handshaking LSTM for remaining useful life prediction. Neurocomputing, 323, 148–156. Retrieved from https://doi.org/10.1016/j.neucom.2018.09.076 doi: 10.1016/j.neucom.2018.09.076
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, PHM 2008. doi: 10.1109/PHM.2008.4711422
Hüllermeier, E., &Waegeman,W. (2021). Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3), 457–506. Retrieved from https://doi.org/10.1007/s10994-021-05946-3 doi: 10.1007/s10994-021-05946-3
Jardine, A. K., Lin, D., & Banjevic, D. (2006, oct). A review on machinery diagnostics and prognostics implementing condition-based maintenance (Vol. 20) (No. 7). doi: 10.1016/j.ymssp.2005.09.012
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-Free Predictive Inference for Regression. Journal of the American Statistical Association, 113(523), 1094–1111. Retrieved from https://doi.org/10.1080/01621459.2017.1307116 doi: 10.1080/01621459.2017.1307116
Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018, may). Machinery health prognostics: A systematic review from data acquisition to RUL prediction (Vol. 104). Academic Press. doi: 10.1016/j.ymssp.2017.11.016
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering and System Safety, 172, 1–11. Retrieved from http://www.elsevier.com/open-access/userlicense/1.0/ doi: 10.1016/j.ress.2017.11.021
Liao, Y., Zhang, L., & Liu, C. (2018, aug). Uncertainty Prediction of Remaining Useful Life Using Long Short-Term Memory Network Based on Bootstrap Method. In 2018 ieee international conference on prognostics and health management, icphm 2018. Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/ICPHM.2018.8448804
Liu, C., Zhang, L., Liao, Y., Wu, C., & Peng, G. (2019). Multiple Sensors Based Prognostics with Prediction Interval Optimization via Echo State Gaussian Process. IEEE Access, 7, 112397–112409. doi: 10.1109/ACCESS.2019.2925634
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on Bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048. doi: 10.1007/s10845-014-0933-4
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, É. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825–2830. Retrieved from http://scikit-learn.sourceforge.net.
Peng, W., Ye, Z. S., & Chen, N. (2020). Bayesian Deep-Learning-Based Health Prognostics Toward Prognostics Uncertainty. IEEE Transactions on Industrial Electronics, 67(3), 2283–2293. doi: 10.1109/TIE.2019.2907440
Rigamonti, M., Baraldi, P., Zio, E., Roychoudhury, I., Goebel, K., & Poll, S. (2018). Ensemble of optimized echo state networks for remaining useful life prediction. Neurocomputing, 281, 121–138. Retrieved from https://doi.org/10.1016/j.neucom.2017.11.062 doi: 10.1016/j.neucom.2017.11.062
Romano, Y., Patterson, E., & Cand`es, E. J. (2019). Conformalized quantile regression. In Advances in neural information processing systems (Vol. 32). Retrieved from https://github.com/yromano/cqr.
Sankararaman, S., & Goebel, K. (2015). Uncertainty in prognostics and systems health management. International Journal of Prognostics and Health Management, 6, 1–14. doi: 10.36001/ijphm.2015.v6i4.2319
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management, phm 2008. doi: 10.1109/PHM.2008.4711414
Tornede, T., Tornede, A., Wever, M., & H¨ullermeier, E. (2021). Coevolution of remaining useful lifetime estimation pipelines for automated predictive maintenance. In Gecco 2021 - proceedings of the 2021 genetic and evolutionary computation conference (pp. 368–376). Retrieved from https://doi.org/10.1145/3449639.3459395 doi: 10.1145/3449639.3459395
Tornede, T., Tornede, A., Wever, M., Mohr, F., & Hüllermeier, E. (2020). AutoML for Predictive Maintenance: One Tool to RUL Them All. In Communications in computer and information science (Vol. 1325, pp. 106–118). Retrieved from https://doi.org/10.1007/978-3-030-66770-2_8 doi: 10.1007/978-3-030-66770-2_8
Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Springer-Verlag. doi: 10.1007/b106715
Wu, Q., Ding, K., & Huang, B. (2020, oct). Approach for fault prognosis using recurrent neural network. Journal of Intelligent Manufacturing, 31(7), 1621–1633. Retrieved from https://link.springer.com/article/10.1007/s10845-018-1428-5 doi: 10.1007/s10845-018-1428-5
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275, 167–179. Retrieved from http://dx.doi.org/10.1016/j.neucom.2017.05.063 doi: 10.1016/j.neucom.2017.05.063
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017, oct). Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2306–2318. doi: 10.1109/TNNLS.2016.2582798
Zhao, Z., Wu, J., Wong, D., Sun, C., & Yan, R. (2020). Probabilistic Remaining Useful Life Prediction Based on Deep Convolutional Neural Network. SSRN Electronic Journal. Retrieved from https://github.com/ZhaoZhibin/Probabilistic RUL Prediction. doi: 10.2139/ssrn.3717738
Section
Technical Papers