Ensemble Learning Based Convolutional Neural Networks for Remaining Useful Life Prediction of Aircraft Engines
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Oct 26, 2023
Thambirajah Ravichandran
Bolun Cui
Sri Namachchivaya
Amar Kumar
Alka Srivatsava
Yuan Liu
Abstract
Remaining useful life (RUL) prediction is an essential task of Prognostics and Health Management (PHM) of aircraft engines performed utilizing the data collected from multiple sensors to ensure their safety. While many studies have been reported on RUL prediction for aircraft engines, only a few of them focus on ensemble learning based convolution neural network (CNN) models for RUL prediction. This paper proposes a new data-driven approach based on a multistage ensemble learning strategy for developing CNN models for RUL prediction of aircraft engines. The proposed approach places a major emphasis on generating diverse CNN models by exploring 2D CNN models and 1D CNN models with multiple channels and developing a multistage ensemble approach employing sparsity promoting model selection and weight learning methods to utilize only a subset of available models thus improving the RUL prediction performance. The effectiveness of the proposed approach is validated using the NASA C-MAPSS dataset for aircraft engines.
How to Cite
Ravichandran, T., Cui, B., Namachchivaya, S., Kumar, A., Srivatsava, A., & Liu, Y. (2023). Ensemble Learning Based Convolutional Neural Networks for Remaining Useful Life Prediction of Aircraft Engines. Annual Conference of the PHM Society, 15(1). https://doi.org/10.36001/phmconf.2023.v15i1.3517
##plugins.themes.bootstrap3.article.details##
Keywords
Remaining useful life, ensemble learning, convolutinal neural networks, hyperparameter optimization
References
Ali, J., Chebel-Morello, B., Saidi, L., Malinowski, S., & Fnaiech, F. (2015). Accurate bearing remaining useful life prediction based on Weibull distribution and artificial neural network. Mechanical Systems and Signal Processing, 56–57, 150–172.
Babu, G.S.; Zhao, P.; Li, X.L. Deep CNN Based Regression Approach for Estimation of Remaining Useful Life. In Proceedings of the International Conference on Database Systems for Advanced Applications, Dallas, TX, USA, 16–19 April 2016.
Breiman, L. (1996a). Bagging predictions. Machine Learning, 24(2), 123–140.
Breiman, L. (1996b). Stacked regressions. Machine Learning, 24(1), 49–64.
Freund, Y., & Schapire, R. E. (1996). Experiments with a New Boosting Algorithm. Proceedings of the 13th International Conference on Machine Learning, 148–156.
Foucart, S., & Koslicki, D. (2014). Sparse recovery by means of nonnegative least squares. IEEE Signal Processing Letters, 21(4), 498–502.
Gebraeel, N. Z., Lawley, M. A., Liu, R., & Parmeshwaran, V. (2004). Residual life predictions from vibration-based degradation signals: a neural network approach. IEEE Transactions on Industrial Electronics, 51, 694–700.
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, 1–6.
Heng, A., Zhan, S., Tan, A., & Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, 23, 724–739.
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Vanhoucke, V., Nguyen, P., Sainath, T., & Kingsbury, B. (2012). Deep Neural Networks for Acoustic Modeling in Speech Recognition. Ieee Signal Processing Magazine, 2(november), 1–27.
Hu, C., Youn, B. D., Wang, P., & Taek Yoon, J. (2012). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Reliability Engineering and System Safety, 103, 120–135.
Jouin, M., Gouriveau, R., Hissel, D., & Zerhouni, N. (2015). Particle filter-based prognostics : review , discussion and perspectives Particle filters - Theory and generalities.
Kalgren, P. W., Byington, C. S., Roemer, M. J., & Watson, M. J. (2006). Defining PHM, A Lexical Evolution of Maintenance and Logistics. 2006 IEEE Autotestcon, 353–358.
Khelif, R., Chebel-Morello, B., Malinowski, S., Laajili, E., Fnaiech, F., & Zerhouni, N. (2017). Direct Remaining Useful Life Estimation Based on Support Vector Regression. IEEE Transactions on Industrial Electronics, 64(3), 2276–2285.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.
Kong, Z., Cui, Y., Xia, Z., & He, L. (2019). Convolution and long short-term memory hybrid deep neural networks for remaining useful life prognostics. Applied Sciences (Switzerland), 9(19).
Lawson, C. L., & Hanson, R. J. (1974). Solving Least Squares Problems. Prentice-Hall, New York.
Li, X., Ding, Q., & Sun, J. Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering and System Safety, 172(December 2017), 1–11.
Li, Z., Goebel, K., & Wu, D. (2019a). Degradation Modeling and Remaining Useful Life Prediction of Aircraft Engines Using Ensemble Learning. Journal of Engineering for Gas Turbines and Power, 141(4), 1–10.
Li, J., Li, X. and He, D. (2019b). A Directed Acyclic Graph Network Combined With CNN and LSTM for Remaining Useful Life Prediction. IEEE Access, 7, pp. 75464–75475.
Miller, A. (2002) Subset Selection in Regression. Second. Boca Raton: Chapman & Hall.
Mo, H., Lucca, F., Malacarne, J., & Iacca, G. (2020).Multi-Head CNN-LSTM with Prediction Error Analysis for Remaining Useful Life Prediction. 2020 27th Conference of Open Innovations Association (FRUCT), pp. 164–171.
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on Bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048.
Pecht, M., & Jie Gu. (2009). Physics-of-failure-based prognostics for electronic products. Transactions of the Institute of Meas. and Control, 31(3–4), 309–322.
Peng, C., Chen, Y., Chen, Q., Tang, Z., Li, L., Gui, W., (2021). A remaining useful life prognosis of turbofan engine using temporal and spatial feature fusion. Sensors (Switzerland), 21(2), pp. 1–21.
Saxena, A., & Goebel, K. (2008). Turbofan Engine Degradation Simulation Data Set. NASA Ames Prognostics Data Repository.
Saxena, Aakanksha, Goebel, K., Simon, D., & Eklund, N. H. W. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. 2008 International Conference on Prognostics and Health Management, 1–9.
Shi, J., Yu, T., Goebel, K., & Wu, D. (2021). Remaining Useful Life Prediction of Bearings Using Ensemble Learning: The Impact of Diversity in Base Learners and Features. Journal of Computing and Information Science in Engineering, 21(2), 1–12.
Schafer, R. (2011). What Is a Savitzky-Golay Filter? [Lecture Notes]. IEEE Signal Processing Magazine - IEEE SIGNAL PROCESS MAG, 28, 111–117.
Schmidhuber, J. (2015). Deep learning in neural networks : An overview. Neural Networks, 61, 85–117.
Vollert, S., & Theissler, A. (2021). Challenges of machine learning-based RUL prognosis: A review on NASA’s C-MAPSS data set. 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA ), 1–8.
Wen, L., Dong, Y., & Gao, L. (2019). A new ensemble residual convolutional neural network for remaining useful life estimation. Mathematical Biosciences and Engineering, 16(2), 862–880.
Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241–259.
Wu, H., Fang, W. Z., Kang, Q., Tao, W. Q., & Qiao, R. (2019). Predicting Effective Diffusivity of Porous Media from Images by Deep Learning. Scientific Reports, 9(1), 1–12.
Yang, H., Zhao, F., Jiang, G., Sun, Z., & Mei, X. (2019). A novel deep learning approach for machinery prognostics based on time windows. Applied Sciences (Switzerland), 9(22).
Yu, K., Wang, D. and Li, H. (2021). A prediction model for remaining useful life of turbofan engines by fusing broad learning system and temporal convolutional network. 8th Int. Conf. Inf., Cybern., Comput. Social Syst. (ICCSS), pp. 134–142.
Zeng, J., & Cheng, Y. (2020). An ensemble learning-based remaining useful life prediction method for aircraft turbine engine. IFAC-PapersOnLine, 53(3), 48–5.
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017). Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2306–2318.
Zhou, Z. H., Wu, J. and Tang, W. (2002). Ensembling neural networks: Many could be better than all. Artificial Intelligence, 137(1–2), pp. 239–263.
Babu, G.S.; Zhao, P.; Li, X.L. Deep CNN Based Regression Approach for Estimation of Remaining Useful Life. In Proceedings of the International Conference on Database Systems for Advanced Applications, Dallas, TX, USA, 16–19 April 2016.
Breiman, L. (1996a). Bagging predictions. Machine Learning, 24(2), 123–140.
Breiman, L. (1996b). Stacked regressions. Machine Learning, 24(1), 49–64.
Freund, Y., & Schapire, R. E. (1996). Experiments with a New Boosting Algorithm. Proceedings of the 13th International Conference on Machine Learning, 148–156.
Foucart, S., & Koslicki, D. (2014). Sparse recovery by means of nonnegative least squares. IEEE Signal Processing Letters, 21(4), 498–502.
Gebraeel, N. Z., Lawley, M. A., Liu, R., & Parmeshwaran, V. (2004). Residual life predictions from vibration-based degradation signals: a neural network approach. IEEE Transactions on Industrial Electronics, 51, 694–700.
Heimes, F. O. (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, 1–6.
Heng, A., Zhan, S., Tan, A., & Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, 23, 724–739.
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Vanhoucke, V., Nguyen, P., Sainath, T., & Kingsbury, B. (2012). Deep Neural Networks for Acoustic Modeling in Speech Recognition. Ieee Signal Processing Magazine, 2(november), 1–27.
Hu, C., Youn, B. D., Wang, P., & Taek Yoon, J. (2012). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Reliability Engineering and System Safety, 103, 120–135.
Jouin, M., Gouriveau, R., Hissel, D., & Zerhouni, N. (2015). Particle filter-based prognostics : review , discussion and perspectives Particle filters - Theory and generalities.
Kalgren, P. W., Byington, C. S., Roemer, M. J., & Watson, M. J. (2006). Defining PHM, A Lexical Evolution of Maintenance and Logistics. 2006 IEEE Autotestcon, 353–358.
Khelif, R., Chebel-Morello, B., Malinowski, S., Laajili, E., Fnaiech, F., & Zerhouni, N. (2017). Direct Remaining Useful Life Estimation Based on Support Vector Regression. IEEE Transactions on Industrial Electronics, 64(3), 2276–2285.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.
Kong, Z., Cui, Y., Xia, Z., & He, L. (2019). Convolution and long short-term memory hybrid deep neural networks for remaining useful life prognostics. Applied Sciences (Switzerland), 9(19).
Lawson, C. L., & Hanson, R. J. (1974). Solving Least Squares Problems. Prentice-Hall, New York.
Li, X., Ding, Q., & Sun, J. Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering and System Safety, 172(December 2017), 1–11.
Li, Z., Goebel, K., & Wu, D. (2019a). Degradation Modeling and Remaining Useful Life Prediction of Aircraft Engines Using Ensemble Learning. Journal of Engineering for Gas Turbines and Power, 141(4), 1–10.
Li, J., Li, X. and He, D. (2019b). A Directed Acyclic Graph Network Combined With CNN and LSTM for Remaining Useful Life Prediction. IEEE Access, 7, pp. 75464–75475.
Miller, A. (2002) Subset Selection in Regression. Second. Boca Raton: Chapman & Hall.
Mo, H., Lucca, F., Malacarne, J., & Iacca, G. (2020).Multi-Head CNN-LSTM with Prediction Error Analysis for Remaining Useful Life Prediction. 2020 27th Conference of Open Innovations Association (FRUCT), pp. 164–171.
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on Bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048.
Pecht, M., & Jie Gu. (2009). Physics-of-failure-based prognostics for electronic products. Transactions of the Institute of Meas. and Control, 31(3–4), 309–322.
Peng, C., Chen, Y., Chen, Q., Tang, Z., Li, L., Gui, W., (2021). A remaining useful life prognosis of turbofan engine using temporal and spatial feature fusion. Sensors (Switzerland), 21(2), pp. 1–21.
Saxena, A., & Goebel, K. (2008). Turbofan Engine Degradation Simulation Data Set. NASA Ames Prognostics Data Repository.
Saxena, Aakanksha, Goebel, K., Simon, D., & Eklund, N. H. W. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. 2008 International Conference on Prognostics and Health Management, 1–9.
Shi, J., Yu, T., Goebel, K., & Wu, D. (2021). Remaining Useful Life Prediction of Bearings Using Ensemble Learning: The Impact of Diversity in Base Learners and Features. Journal of Computing and Information Science in Engineering, 21(2), 1–12.
Schafer, R. (2011). What Is a Savitzky-Golay Filter? [Lecture Notes]. IEEE Signal Processing Magazine - IEEE SIGNAL PROCESS MAG, 28, 111–117.
Schmidhuber, J. (2015). Deep learning in neural networks : An overview. Neural Networks, 61, 85–117.
Vollert, S., & Theissler, A. (2021). Challenges of machine learning-based RUL prognosis: A review on NASA’s C-MAPSS data set. 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA ), 1–8.
Wen, L., Dong, Y., & Gao, L. (2019). A new ensemble residual convolutional neural network for remaining useful life estimation. Mathematical Biosciences and Engineering, 16(2), 862–880.
Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241–259.
Wu, H., Fang, W. Z., Kang, Q., Tao, W. Q., & Qiao, R. (2019). Predicting Effective Diffusivity of Porous Media from Images by Deep Learning. Scientific Reports, 9(1), 1–12.
Yang, H., Zhao, F., Jiang, G., Sun, Z., & Mei, X. (2019). A novel deep learning approach for machinery prognostics based on time windows. Applied Sciences (Switzerland), 9(22).
Yu, K., Wang, D. and Li, H. (2021). A prediction model for remaining useful life of turbofan engines by fusing broad learning system and temporal convolutional network. 8th Int. Conf. Inf., Cybern., Comput. Social Syst. (ICCSS), pp. 134–142.
Zeng, J., & Cheng, Y. (2020). An ensemble learning-based remaining useful life prediction method for aircraft turbine engine. IFAC-PapersOnLine, 53(3), 48–5.
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017). Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2306–2318.
Zhou, Z. H., Wu, J. and Tang, W. (2002). Ensembling neural networks: Many could be better than all. Artificial Intelligence, 137(1–2), pp. 239–263.
Section
Technical Research Papers
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.