LSTM and Transformers based methods for Remaining Useful Life Prediction considering Censored Data
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Jul 4, 2025
Jean-Pierre Noot
Mikaël Martin
Etienne Birmele
Abstract
Predictive maintenance deals with the timely replacement of industrial components relatively to their failure. It allows to prevent shutdowns as in reactive maintenance and reduces the costs compared to preventive maintenance. As a consequence, Remaining Useful Life (RUL) prediction of industrial components has become a key challenge for condition-based monitoring. In many applications, in particular those for which preventive maintenance is the general rule, the prediction problem is made harder by the rarity of failing instances. Indeed, the interruption of data acquisition before the occurrence of the event of interest leads to right censored data. Recent deep-learning architectures, that show the best results of the literature for complete-life data, most often do not consider censoring, even though its rate in the industrial environment may be high.
The present article introduces a method which considers censored data for the Dual Aspect Self-Attention based on a Transformer proposed by (Z. Zhang, Song, & Li, 2022), and puts it into competition a modified version of the ordinal regression-based LSTMof (Vishnu, Malhotra, Vig, & Shroff, 2019). The evaluation of the resulting method on the CMAPSS and N-CMAPSS benchmark dataset shows that it is competitive compared to the state-of-the-art RUL prediction methods for a low censoring rate and more efficient for a high rate of censoring in large enough data sets. Finally, conformal prediction is used to estimate confidence intervals for the predictions.
##plugins.themes.bootstrap3.article.details##
Keywords
RUL estimation, Deep Learning, Transformers, LSTM, Conformal Prediciton, Censored data
References
Angelopoulos, A. N., Bates, S., et al. (2023). Conformal prediction: A gentle introduction. Foundations and Trends® in Machine Learning, 16(4), 494–591.
Arena, F., Collotta, M., Luca, L., Ruggieri, M., & Termine, F. G. (2021). Predictive maintenance in the automotive sector: A literature review. Mathematical and Computational Applications, 27(1), 2.
Arias Chao, M., Kulkarni, C., Goebel, K., & Fink, O. (2021). Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics. Data, 6(1), 5.
Boutrous, K., Bessa, I., Puig, V., Nejjari, F., & Palhares, R. M. (2022). Data-driven prognostics based on evolving fuzzy degradation models for power semiconductor devices. In Phm society european conference (Vol. 7, pp. 68–77).
Chaoub, A., Voisin, A., Cerisara, C., & Iung, B. (2021). Learning representations with end-to-end models for improved remaining useful life prognostics. arXiv preprint arXiv:2104.05049.
Chen, C., Liu, Y., Wang, S., Sun, X., Di Cairano-Gilfedder, C., Titmus, S., & Syntetos, A. A. (2020). Predictive maintenance using cox proportional hazard deep learning. Advanced Engineering Informatics, 44, 101054.
Garay, J. M., & Diedrich, C. (2019). Analysis of the applicability of fault detection and failure prediction based on unsupervised learning and monte carlo simulations for real devices in the industrial automobile production. In 2019 ieee 17th international conference on industrial informatics (indin) (Vol. 1, pp. 1279–1284).
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. In 2008 international conference on prognostics and health management (pp. 1–6).
Javanmardi, S., & Hüllermeier, E. (2023). Conformal prediction intervals for remaining useful lifetime estimation. arXiv preprint arXiv:2107.07511.
Jean-Pierre, N., Birmelé, E., & François, R. (2024). Lstm and transformers based methods for remaining useful life prediction considering censored data. In Phm society european conference (Vol. 8, pp. 10–10).
Jeong, K., & Choi, S. (2019). Model-based sensor fault diagnosis of vehicle suspensions with a support vector machine. International Journal of Automotive Technology, 20, 961–970.
Kong, Y., Abdullah, S., Schramm, D., Omar, M., & Haris, S. (2019). Development of multiple linear regression-based models for fatigue life evaluation of automotive coil springs. Mechanical Systems and Signal Processing, 118, 675–695.
Lai, Z., Liu, M., Pan, Y., & Chen, D. (2022). Multi-dimensional self attention based approach for remaining useful life estimation. arXiv preprint arXiv:2212.05772.
Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020). Remaining useful life prediction using multi-scale deep convolutional neural network. Applied Soft Computing, 89, 106113.
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1–11.
Patil, S., Patil, A., Handikherkar, V., Desai, S., Phalle, V. M., & Kazi, F. S. (2018). Remaining useful life (rul) prediction of rolling element bearing using random forest and gradient boosting technique. In Asme international mechanical engineering congress and exposition (Vol. 52187, p. V013T05A019).
Sateesh Babu, G., Zhao, P., & Li, X.-L. (2016). Deep convolutional neural network based regression approach for estimation of remaining useful life. In Database systems for advanced applications: 21st international conference, dasfaa 2016, dallas, tx, usa, april 16-19, 2016, proceedings, part i 21 (pp. 214–228).
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management (pp. 1–9).
Song, Y., Gao, S., Li, Y., Jia, L., Li, Q., & Pang, F. (2020). Distributed attention-based temporal convolutional network for remaining useful life prediction. IEEE Internet of Things Journal, 8(12), 9594–9602.
Tinga, T., & Loendersloot, R. (2019). Physical model-based prognostics and health monitoring to enable predictive maintenance. Predictive Maintenance in Dynamic Systems: Advanced Methods, Decision Support Tools and Real-World Applications, 313–353.
Vasavi, S., Aswarth, K., Pavan, T. S. D., & Gokhale, A. A. (2021). Predictive analytics as a service for vehicle health monitoring using edge computing and ak-nn algorithm. Materials Today: Proceedings, 46, 8645–8654.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., . . . Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Vishnu, T., Malhotra, P., Vig, L., & Shroff, G. (2019). Data-driven prognostics with predictive uncertainty estimation using ensemble of deep ordinal regression models. International Journal of Prognostics and Health Management, 10(4).
Yang, Z., Kanniainen, J., Krogerus, T., & Emmert-Streib, F. (2022). Prognostic modeling of predictive maintenance with survival analysis for mobile work equipment. Scientific Reports, 12(1), 8529. Retrieved from https://doi.org/10.1038/s41598-022-12572-z doi: 10.1038/s41598-022-12572-z
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2016). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE transactions on neural networks and learning systems, 28(10), 2306–2318.
Zhang, Z., Song, W., & Li, Q. (2022). Dual aspect self-attention based on transformer for remaining useful life prediction. IEEE Transactions on Instrumentation and Measurement, 71, 1–11.
Zhao, C., Huang, X., Li, Y., & Yousaf Iqbal, M. (2020). A double-channel hybrid deep neural network based on cnn and bilstm for remaining useful life prediction. Sensors, 20(24), 7109.
Zhao, Y., Liu, P., Wang, Z., & Hong, J. (2017). Electric vehicle battery fault diagnosis based on statistical method. Energy Procedia, 105, 2366–2371.
Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017). Long short-term memory network for remaining useful life estimation. In 2017 ieee international conference on prognostics and health management (icphm) (pp. 88–95).
Arena, F., Collotta, M., Luca, L., Ruggieri, M., & Termine, F. G. (2021). Predictive maintenance in the automotive sector: A literature review. Mathematical and Computational Applications, 27(1), 2.
Arias Chao, M., Kulkarni, C., Goebel, K., & Fink, O. (2021). Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics. Data, 6(1), 5.
Boutrous, K., Bessa, I., Puig, V., Nejjari, F., & Palhares, R. M. (2022). Data-driven prognostics based on evolving fuzzy degradation models for power semiconductor devices. In Phm society european conference (Vol. 7, pp. 68–77).
Chaoub, A., Voisin, A., Cerisara, C., & Iung, B. (2021). Learning representations with end-to-end models for improved remaining useful life prognostics. arXiv preprint arXiv:2104.05049.
Chen, C., Liu, Y., Wang, S., Sun, X., Di Cairano-Gilfedder, C., Titmus, S., & Syntetos, A. A. (2020). Predictive maintenance using cox proportional hazard deep learning. Advanced Engineering Informatics, 44, 101054.
Garay, J. M., & Diedrich, C. (2019). Analysis of the applicability of fault detection and failure prediction based on unsupervised learning and monte carlo simulations for real devices in the industrial automobile production. In 2019 ieee 17th international conference on industrial informatics (indin) (Vol. 1, pp. 1279–1284).
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. In 2008 international conference on prognostics and health management (pp. 1–6).
Javanmardi, S., & Hüllermeier, E. (2023). Conformal prediction intervals for remaining useful lifetime estimation. arXiv preprint arXiv:2107.07511.
Jean-Pierre, N., Birmelé, E., & François, R. (2024). Lstm and transformers based methods for remaining useful life prediction considering censored data. In Phm society european conference (Vol. 8, pp. 10–10).
Jeong, K., & Choi, S. (2019). Model-based sensor fault diagnosis of vehicle suspensions with a support vector machine. International Journal of Automotive Technology, 20, 961–970.
Kong, Y., Abdullah, S., Schramm, D., Omar, M., & Haris, S. (2019). Development of multiple linear regression-based models for fatigue life evaluation of automotive coil springs. Mechanical Systems and Signal Processing, 118, 675–695.
Lai, Z., Liu, M., Pan, Y., & Chen, D. (2022). Multi-dimensional self attention based approach for remaining useful life estimation. arXiv preprint arXiv:2212.05772.
Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020). Remaining useful life prediction using multi-scale deep convolutional neural network. Applied Soft Computing, 89, 106113.
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1–11.
Patil, S., Patil, A., Handikherkar, V., Desai, S., Phalle, V. M., & Kazi, F. S. (2018). Remaining useful life (rul) prediction of rolling element bearing using random forest and gradient boosting technique. In Asme international mechanical engineering congress and exposition (Vol. 52187, p. V013T05A019).
Sateesh Babu, G., Zhao, P., & Li, X.-L. (2016). Deep convolutional neural network based regression approach for estimation of remaining useful life. In Database systems for advanced applications: 21st international conference, dasfaa 2016, dallas, tx, usa, april 16-19, 2016, proceedings, part i 21 (pp. 214–228).
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management (pp. 1–9).
Song, Y., Gao, S., Li, Y., Jia, L., Li, Q., & Pang, F. (2020). Distributed attention-based temporal convolutional network for remaining useful life prediction. IEEE Internet of Things Journal, 8(12), 9594–9602.
Tinga, T., & Loendersloot, R. (2019). Physical model-based prognostics and health monitoring to enable predictive maintenance. Predictive Maintenance in Dynamic Systems: Advanced Methods, Decision Support Tools and Real-World Applications, 313–353.
Vasavi, S., Aswarth, K., Pavan, T. S. D., & Gokhale, A. A. (2021). Predictive analytics as a service for vehicle health monitoring using edge computing and ak-nn algorithm. Materials Today: Proceedings, 46, 8645–8654.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., . . . Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Vishnu, T., Malhotra, P., Vig, L., & Shroff, G. (2019). Data-driven prognostics with predictive uncertainty estimation using ensemble of deep ordinal regression models. International Journal of Prognostics and Health Management, 10(4).
Yang, Z., Kanniainen, J., Krogerus, T., & Emmert-Streib, F. (2022). Prognostic modeling of predictive maintenance with survival analysis for mobile work equipment. Scientific Reports, 12(1), 8529. Retrieved from https://doi.org/10.1038/s41598-022-12572-z doi: 10.1038/s41598-022-12572-z
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2016). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE transactions on neural networks and learning systems, 28(10), 2306–2318.
Zhang, Z., Song, W., & Li, Q. (2022). Dual aspect self-attention based on transformer for remaining useful life prediction. IEEE Transactions on Instrumentation and Measurement, 71, 1–11.
Zhao, C., Huang, X., Li, Y., & Yousaf Iqbal, M. (2020). A double-channel hybrid deep neural network based on cnn and bilstm for remaining useful life prediction. Sensors, 20(24), 7109.
Zhao, Y., Liu, P., Wang, Z., & Hong, J. (2017). Electric vehicle battery fault diagnosis based on statistical method. Energy Procedia, 105, 2366–2371.
Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017). Long short-term memory network for remaining useful life estimation. In 2017 ieee international conference on prognostics and health management (icphm) (pp. 88–95).
Section
Technical Papers