Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Among the various data-driven approaches used for RUL prediction, Recurrent Neural Networks (RNNs) have certain prima facie advantages over other approaches because the connections between internal nodes form directed cycles, thus creating internal states which enables the network to encapsulate dynamic temporal behavior and also to properly handle the noise affecting the collected signals. However, the application of traditional RNNs is limited by the difficulty of optimizing their numerous internal parameters and the significant computational effort associated with the training process. In this work, we explore the use of the Echo State Network (ESN), a relatively new type of Recurrent Neural Network (RNN). One of the main advantages of ESN is the training procedure, which is based on a simple linear regression. Unlike traditional RNNs, ESNs can be trained with fairly little computational effort, while still providing the generalization capability characteristic of RNNs. In this paper, we use Differential Evolution (DE) for the optimization of the ESN architecture for RUL prediction of a turbofan engine working under variable operating conditions. A procedure for pre-processing of the monitored signals and for identification of the onset of acceleration of degradation (i.e., the so-called elbow point in the degradation trend) will be shown. The datasets used to validate the approach have been taken from the NASA Ames Prognostics CoE Data Repository. These datasets were generated using a turbofan engine simulator, based on a detailed physical model that allows input variations of health-related parameters under variable operating conditions and records values from some specific sensor measurements. The results obtained on these data confirm the ESN’s capability to provide accurate RUL predictions.
How to Cite
##plugins.themes.bootstrap3.article.details##
data driven prognostics, recurrent neural networks, CMAPPS datasets, echo state networks, differential evolution
Bonissone, P.P., Xue, F., Subbu, R., (2011). Fast meta-models for local fusion of multiple predictive models. Applied Soft Computing Journal, 11 (2), pp. 1529-1539.
Chen, S.J., Hwang, C.L., (1992). Fuzzy Multiple Attribute Decision Making: Methods and Applications. Springer- Verlag, Berlin.
Coble, J. B. (2010). Merging data sources to predict remaining useful life–an automated method to identify prognostic parameters. Doctoral dissertation.
Daigle, M.J., Roychoudhury, I., Biswas, G., Koutsoukos, X.D., Patterson-Hine, A., Poll, S., (2010). A comprehensive diagnosis methodology for complex hybrid systems: A case study on spacecraft power distribution systems. IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans, 40 (5), art. no. 5504182, pp. 917-931.
Ferreira, A.A., Ludermir, T.B. (2009). Genetic algorithm for reservoir computing optimization. Proceedings of the international joint conference on neural networks – IJCNN 2009, Atlanta (pp. 811–815).
Ferreira, A.A., Ludermir, T.B., De Aquino, R.R.B., (2013). An approach to reservoir computing design and training. Expert Systems with Applications, 40 (10), pp. 4172-4182.
Fink, O., Zio, E., Weidmann, U., (2013). Predicting time series of railway speed restrictions with time-dependent machine learning techniques. Expert Systems with Applications, 40 (15), pp. 6033-6040.
Fink, O., Weidmann, U., Zio, E., (2014). Extreme learning machines for predicting operation disruption events in railway systems. Safety, Reliability and Risk Analysis: Beyond the Horizon - Proceedings of the European Safety and Reliability Conference, ESREL 2013, pp. 1781-1787.
Frederick, D., DeCastro, J., Litt, J., (2007). User's Guide for the Commercial Modular Aero-Propulsion System Simulation (CMAPSS). NASA/ARL, Technical Manual TM 2007-215026.
Gasperin, M., Boskoski, P., Juricic, D., (2011). Model-based prognostics under non-stationary operating conditions. In Annual Conference of the Prognostics and Health Management Society (pp. 831-853).
Heimes, F.O., (2008). Recurrent neural networks for remaining useful life estimation. 2008 International Conference on Prognostics and Health Management, PHM 2008, art. n. 4711422.
Hu, Y., (2015). Development of prognostics and health management methods for engineering systems operating in evolving environments. PhD Thesis, Politecnico di Milano, 2015.
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., (2006). Extreme learning machine: Theory and applications. Neurocomputing, 70 (1-3), pp. 489-501.
Jaeger, H., (2001). The echo state approach to analyzing and training recurrent neural networks. Technical Report GMD Report 148, German National Research Center for Information Technology.
Jaeger, H., (2002). A Tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the Echo state network approach. Technical Report GMD Report 159, German National Research Center for Information Technology.
Jaeger, H., Haas, H., (2004). Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science, 304 (5667), pp. 78-80.
Li, D., Han, M., Wang, J., (2012). Chaotic time series prediction based on a novel robust echo state network. IEEE Transactions on Neural Networks and Learning Systems, 23 (5), art. no. 6177672, pp. 787-797.
Li, X., Qian, J., Wang, G. (2013). Fault prognostic based on hybrid method of state judgment and regression. Advances in Mechanical Engineering, 2013(149562), 1-10.
Liu, J., Saxena, A., Goebel, K., Saha, B., Wang, W., (2010). An adaptive recurrent neural network for remaining useful life prediction of lithium-ion batteries. Annual Conference of the Prognostics and Health Management Society, PHM 2010.
Lukoševičius, M., Jaeger, H., (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3 (3), pp. 127-149.
Malhi, A., Yan, R., Gao, R.X., (2011). Prognosis of defect propagation based on recurrent neural networks. IEEE Transactions on Instrumentation and Measurement, 60 (3), art. no. 5710193, pp. 703-711.
Morando, S., Jemei, S., Gouriveau, R., Zerhouni, N., Hissel, D., (2013). Fuel Cells prognostics using echo state network. IECON Proceedings (Industrial Electronics Conference), art. no. 6699377, pp. 1632-1637.
Moustapha, A. I., Selmic, R. R., (2008). Wireless sensor network modeling using modified recurrent neural networks: Application to fault detection. IEEE Trans. Instrum. Meas., vol. 57, no. 5, pp. 981–988.
Olivares, B.E., Cerda Muñoz, M.A., Orchard, M.E., Silva, J.F., (2013). Particle-filtering-based prognosis framework for energy storage devices with a statistical characterization of state-of-health regeneration phenomena. IEEE Transactions on Instrumentation and Measurement, 62 (2), art. no. 6302189, pp. 364-376.
Opricovic, S., Tzeng, G.-H., (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156 (2), pp. 445-455.
Pecht, M.G., (2008). Prognostics and Health Management of Electronics. Prognostics and Health Management of Electronics, pp. 1-315.
Peng, Y., Wang, H., Wang, J., Liu, D., Peng, X., (2012a). A modified echo state network based remaining useful life estimation approach. PHM 2012 - 2012 IEEE Int. Conf. on Prognostics and Health Management: Enhancing Safety, Efficiency, Availability, and Effectiveness of Systems Through PHM Technology and Application, Conference Program, art. no. 6299524.
Peng, Y., Xu, Y., Liu, D., Peng, X. (2012b). Sensor selection with grey correlation analysis for remaining useful life evaluation. In Annual Conference of the PHM Society.
Qu, J., Zuo, M.J., (2012). An LSSVR-based algorithm for online system condition prognostics. Expert Systems with Applications, 39 (5), pp. 6089-6102.
Rabin, M.J.A., Hossain, M.S., Ahsan, M.S., Mollah, M.A.S., Rahman, M.T., (2013). Sensitivity learning oriented non-monotonic multi reservoir echo state network for short-term load forecasting. 2013 International Conference on Informatics, Electronics and Vision, ICIEV 2013, art. no. 6572692.
Ramasso E., Saxena, A., (2014). Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets. International Journal of Prognostics and Health Management, , 2014, 5 (2), pp.1-15.
Ramasso, E., (2014). Investigating computational geometry for failure prognostics. International Journal of Prognostics and Health Management, 2014, 5(5), pp 1-18.
Samanta, B., Al-Balushi, K., (2003). Artificial neural network based fault diagnostics of rolling element bearings using time-domain features. Mechanical System Signal Processing, vol. 17, no. 2, pp. 317–328.
Saxena, A., Goebel, K., Simon, D., Eklund, N., (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. 2008 International Conference on Prognostics and Health Management, PHM 2008, art. no. 4711414.
Saxena, A., Celaya, J., Saha, B., Saha, S., Goebel, K., (2010). Metrics for offline evaluation of prognostic performance. International Journal of Prognostics and Health Management, 1 (1).
Shi, Z., Han, M., (2007). Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks, 18 (2), pp. 359-372.
Storn, R., Price, K., (1997). Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization, 11 (4), pp. 341-359.
Tse, P.W., Atherton, D.P., (1999). Prediction of machine deterioration using vibration based fault trends and recurrent neural networks. Journal of Vibration and Acoustics, Transactions of the ASME, 121 (3), pp. 355-362.
Vachtsevanos, G., Lewis, F. L., Roemer, M., Hess, A., Wu, B., (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems, Wiley, New York.
Vukicevic, A.M., Jovicic, G.R., Stojadinovic, M.M., Prelevic, R.I., Filipovic, N.D., (2014). Evolutionary assembled neural networks for making medical decisions with minimal regret: Application for predicting advanced bladder cancer outcome. Expert Systems with Applications, 41 (18), pp. 8092-8100.
Wang, T., (2010). Trajectory similarity based prediction for remaining useful life estimation. PhD Thesis, University of Cincinnati, 2010.
Yan, T., Duwu, D., Yongqing, T., (2007). A new evolutionary neural network algorithm based on improved genetic algorithm and its application in power transformer fault diagnosis. 2nd International Conference on Bio-Inspired Computing: Theories and Applications, BICTA 2007, art. no. 4806406, pp. 1-5.
Yildiz, I.B., Jaeger, H., Kiebel, S.J., (2012). Re-visiting the echo state property. Neural Networks, 35, pp. 1-9.
Zio, E., Broggi, M., Pedroni, N., (2009). Nuclear reactor dynamics on-line estimation by Locally Recurrent Neural Networks. Progress in Nuclear Energy, 51 (3), pp. 573-581.
Zio, E., Di Maio, F., Stasi, M., (2010). A data-driven approach for predicting failure scenarios in nuclear systems. Annals of Nuclear Energy, 37 (4), pp. 482-491.
Zitzler, E., Thiele, L., (1999). Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 3 (4), pp. 257-271.
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.