Prognostic of RUL based on Echo State Network Optimized by Artificial Bee Colony
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Nov 11, 2020
Edgar J. Amaya
Alberto J. Alvares
Abstract
Prognostic is an engineering technique used to predict the future health state or behavior of an equipment or system. In this work, a data-driven hybrid approach for prognostic is presented. The approach based on Echo State Network (ESN) and Artificial Bee Colony (ABC) algorithm is used to predict machine’s Remaining Useful Life (RUL). ESN is a new paradigm that establishes a large space dynamic reservoir to replace the hidden layer of Recurrent Neural Network (RNN). Through the application of ESN is possible to overcome the shortcomings of complicated computing and difficulties in determining the network topology of traditional RNN. This approach describes the ABC algorithm as a tool to set the ESN with optimal parameters. Historical data collected from sensors are used to train and test the proposed hybrid approach in order to estimate the RUL. To evaluate the proposed approach, a case study was carried out using turbofan engine signals show that the proposed method can achieve a good collected from physical sensors (temperature, pressure, speed, fuel flow, etc.). The experimental results using the engine data from NASA Ames Prognostics Data Repository RUL estimation precision. The performance of this model was compared using prognostic metrics with the approaches that use the same dataset. Therefore, the ESN-ABC approach is very promising in the field of prognostics of the RUL.
##plugins.themes.bootstrap3.article.details##
Keywords
turbofan engines, particle swarm optimization, Data-driven prognostics, failure prognostics, RUL estimation, echo state networks, Artificial bee Colony
References
Boccato, L. (2013). Novas propostas e aplicações de redes neurais com estados de eco. Tese de doutorado, Faculdade de Engenharia Elétrica e de Computação, Universidade Estadual de Campinas, Campinas, SP, Brasil.
Bush, K., & Tsendjav, B. (2005). Improving the richness of echo state features using next ascent local search. In Proceedings of the artificial neural networks in engineering conference (pp. 227–232).
Butani, R.C., Gajjar, B.D. and Thakker, R.A. (2011). Performance evaluation of Particle SwarmOptimization (PSO) and Artificial Bee Colony (ABC) Algorithm. International Conference on Advanced Computing, Communication and Networks.
Compare, M. and Zio, E. (2014). Predictive Maintenance by Risk Sensitive Particle Filtering. Reliability IEEE Transactions on, vol.63, no.1, pp.134-143.
Daroogheh, N., Meskin, N. and Khorasani, K. (2014). A novel particle filter parameter prediction scheme for failure prognosis. American Control Conference (ACC), vol., no., pp.1735-1742.
Dong, M. and He, D., (2007). Hidden semi-Markov model-based methodology for multisensory equipment health diagnosis and prognosis. European Journal of Operational Research 178 (3), 858–878.
Ferreira, A. A., Ludermir, T.B., Aquino, R., Lira, M.M. and Neto, O.N. (2008). Investigating the use of reservoir computing for forecasting the hourly wind speed in short-term. In International Joint Conference on Neural Networks - IJCNN, pages 1950–1957, Hong Kong.
Ferreira, A. A. e Ludermir, T.B. (2009). Genetic algorithm for reservoir computing optimization. Neural Networks. IJCNN - International Joint Conference on, vol., no., pp.811-815.
Ferreira, A. A., Ludermir, T.B. and Aquino, R. (2013). An approach to reservoir computing design and training. Expert Systems with Applications, Volume 40, Issue 10, pp. 4172-4182.
Ferreiro, S., Arnaiz, A., Sierra, B. and Irigoien, I. (2012). Application of Bayesian networks in prognostics for a new Integrated Vehicle Health Management concept. Expert Systems with Applications, Volume 39, Issue 7, Pages 6402-6418.
Frederick, D., De Castro, J. and Litt, J. (2007). “User’s Guide for the Commercial Modular Aero-Propulsion System Simulation (CMAPSS).” NASA/ARL.
Heng, A., Zhang, S. Tan, A. and Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, Volume 23, Issue 3, pp. 724-739.
Hu, C., Youn, B.D. and Wang, P. (2011). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Prognostics and Health Management (PHM), IEEE Conference on , vol., no., pp.1-10.
Hossain, M.S. and El-Shafie, A. (2014). Evolutionary techniques versus swarm intelligences: application in reservoir release optimization. Neural Computing and Applications, volume 24, number 7-8, pp.1583-1594.
Gasperin, M., Juricic, Baskoski, P. and Jozef, V. (2011). Model-based prognostics of gear health using stochastic dynamic models. Mechanical Systems and Signal Processing 25, pp. 537–538.
Ishii, K., van der Zant, T., Becanovic, V., & Ploger, P. (2004). Identification of motion with echo state network. In Proceedings of the OCEANS MTS/IEEE–TECHNOOCEAN conference, Vol. 3, pp. 1205–1210.
Jaeger, H (2001). The Echo state approach to analyzing and training recurrent neural networks. Technical Report GMD Report 148, German National Research Center for Information Technology, URL www.faculty.jacobs-university.de/hjaeger/pubs/ EchoStatesTechRep.pdf.
Jaeger, H (2002a). Short-term memory in echo state networks. Technical report, GDM 152, German National Resource Center for Information Technology.
Jaeger, H (2002b). A tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the echo state network approach. Technical Report GMD Report 159, German National Research Center for Information Technology, URL www.faculty.jacobs-university.de/hjaeger/pubs/ESNTutorial Rev.pdf.
Jaeger, H. (2003). Adaptive nonlinear system identification with echo state networks, in Advances in Neural Information Processing Systems, S. Becker, S. Thrun, K. Obermayer, Eds., Cambridge, MA: MIT Press, pp.593-600.
Jaeger, H. and Haas, H. (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 304(5667): pp. 78-80, URL www.faculty.jacobs-university.de/ hjaeger/pubs/ ESNScience04.pdf.
Karaboga, D. and Basturk, B. (2007). A powerful and Efficient Algorithm for Numerical Function Optimization: Artificial Bee Colony (ABC) Algorithm. Journal of Global Optimization, Volume: 39, Issue: 3, pp. 459-171.
Karaboga, D. and Akay, B. (2009). A comparative study of Artificial Bee Colony algorithm. Applied Mathematics and Computation, Volume 214, Issue 1, Pages 108-132.
Karaboga, D. and Ozturk, C. (2009). Neural Networks Training by Artificial Bee Colony Algorithm on Pattern Classification. Neural Network World, 19 (3), 279-292.
Karaboga, D. and Ozturk, C. (2010). Fuzzy clustering with artificial bee colony algorithm. Scientific Research and Essays, Volume: 5 Issue: 14 Pages: 1899-1902.
Karaboga, D. (2010). Artificial bee colony algorithm. Scholarpedia, 5(3):6915.
Kumar, S., Torres, M., Chan, Y.C. and Pecht, M. (2008). A hybrid prognostics methodology for electronic products. Neural Networks. IJCNN. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, vol., no., pp. 3479-3485.
Li, D., Wang, W., and Ismail, F. (2013). Enhanced fuzzy-filtered neural networks for material fatigue prognosis. Applied Soft Computing, Vol. 13, No. 1, pp. 283–291.
Liao, L. and Kottig, F. (2014). Review of Hybrid Prognostics Approaches for Remaining Useful Life Prediction of Engineered Systems, and an Application to Battery Life Prediction. Reliability, IEEE Transactions on, vol.63, no.1, pp. 191-207.
Liu, K., Gebraeel, N. Z. and Shi, J. (2013). A data-level fusion model for developing composite health indices for degradation modeling and prognostic analysis. IEEE Trans. on Automation Science and Engineering.
Lukosevicius, M. and Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3, pp.127–149.
Maass, W., Natschlager, T. and Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531–2560.
Natschlager, T., Maass, W. and Markram, H. (2002). The ‘liquid computer’: A novel strategy for real-time computing on time series. Special Issue on Foundations of Information Processing of TELEMATIK, 8(1):39–43.
Pecht, M. (2008). Prognostics and Health Management of Electronics. John Wiley, New Jersey.
Pecht, M. and Jaai, R. (2010). A prognostics and health management roadmap for information and electronics-rich system. Microelectronics Reliability 50, pp. 317–323.
Peng, Y., Wang, H., Wang, J., Liu, D. and Peng, X. (2012). A modified echo state network based remaining useful life estimation approach. Prognostics and Health Management (PHM), IEEE Conference on, vol., no., pp.1-7.
Pla, A., López, B., Gay, P. and Pous, C. (2013). eXiT*CBR.v2: Distributed case-based reasoning tool for medical prognosis, Decision Support Systems, Volume 54, Issue 3, pp. 1499-1510.
Prechelt. L. (1994). Proben1 - a set of neural network benchmark problems and benchmarking rules. Technical report, 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany.
Ramasso, E. and Saxena, A. (2014). Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets,” International Journal of Prognostics and Health Management, v. 5, n. 2, p. 1–15.
Saxena, A. and Goebel, K. (2008). Turbofan Engine Degradation Simulation Data Set, NASA Ames Prognostics Data Repository (http://ti.arc.nasa.gov/ tech/dash/pcoe/prognostic-data-repository), NASA Ames Research Center, Moffett Field, CA.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S. and Schwabacher, M. (2008a). Metrics for Evaluating Performance of Prognositc Techniques. International Conference on Prognostics and Health Management, Denver, CO.
Saxena, A., Goebel, K., Simon, D. and Eklund, N. (2008b). Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, p. 1–9. International Conference On Prognostics And Health Management, Denver, CO, Oct 06-09.
Saxena, A., Celaya, J., Saha, B., Saha, S. and Goebel, K. (2010). Metrics for offline evaluation of prognostic performance. International Journal of Prognostics and Health Management.
Schrauwen, B., Verstraeten, D. and Campenhout, J.V. (2007). An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th European Symposium on Artificial Neural Networks, pp. 471–482.
Shankar, B. (2015). Remaining Useful Life Prediction through Failure Probability Computation for Condition-based Prognostics. In Proceedings of the Annual Conference of the Prognostics and Health Management Society.
Song, Q., Zhao, X. Feng, Z. An, Y. and Song, B. (2011). Hourly electric load forecasting algorithm based on echo state neural network, Control and Decision Conference (CCDC), 2011 Chinese, vol., no., pp.3893, 3897, 23-25 May.
Steil, J.J. (2004). Backpropagation-decorrelation: online recurrent learning with O(N) complexity. In Proc. IJCNN.
ToolboxABC (2015). MATLAB code v2 of the basic ABC algorithm. Available in: < http://mf.erciyes.edu.tr/ abc>. Access in: march 2015.
ToolboxESN (2015). Simple and very simple Matlab toolbox for Echo State Networks. Available in:. Access in: march 2015.
Turanoglu, E., Ozceylan, E. and Kiran, M.S. (2011). Particle Swarm Optimization and Artificial Bee Colony Approaches to Optimize of Single Input-Output Fuzzy Membership Functions. Proceedings of the 41st International Conference on Computers & Industrial Engineering.
Vachtsevanos, G., Lewis, F. L., Roemer, M., Hess, A. and Wu, B. (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems. 1st ed. Hoboken, New Jersey: John Wiley & Sons, Inc.
Verstraeten, D., Schrauwen, B., D’Haene, M. and Stroobandt, D. (2007). An experimental unification of reservoir computing methods. Neural Networks, 20:391–403.
Wang, T., (2010). Trajectory Similarity Based Prediction for Remaining Useful Life Estimation. Doctoral dissertation. University of Cincinnati.
Wang, W., Zhang, W., (2008). An asset residual life prediction model based on expert judgments. European Journal of Operational Research 188, pp. 496–505.
Weiming, W., Bing, L., Min, L. and Houjun, W. (2014). Prognostics of Lithium-Ion Batteries Based on the Verhulst Model, Particle Swarm Optimization and Particle Filter. Instrumentation and Measurement, IEEE Transactions on, vol.63, no.1, pp.2-17.
Zhang, Y., Zhao, X., Liu, W., Zhang, J., Jia, Y. and Feng, T. (2011). Research on gearbox wearing prognosis based on Gamma-State Space Model. Reliability, Maintainability and Safety (ICRMS), 9th International Conference on, vol., no., pp. 279-283.
Bush, K., & Tsendjav, B. (2005). Improving the richness of echo state features using next ascent local search. In Proceedings of the artificial neural networks in engineering conference (pp. 227–232).
Butani, R.C., Gajjar, B.D. and Thakker, R.A. (2011). Performance evaluation of Particle SwarmOptimization (PSO) and Artificial Bee Colony (ABC) Algorithm. International Conference on Advanced Computing, Communication and Networks.
Compare, M. and Zio, E. (2014). Predictive Maintenance by Risk Sensitive Particle Filtering. Reliability IEEE Transactions on, vol.63, no.1, pp.134-143.
Daroogheh, N., Meskin, N. and Khorasani, K. (2014). A novel particle filter parameter prediction scheme for failure prognosis. American Control Conference (ACC), vol., no., pp.1735-1742.
Dong, M. and He, D., (2007). Hidden semi-Markov model-based methodology for multisensory equipment health diagnosis and prognosis. European Journal of Operational Research 178 (3), 858–878.
Ferreira, A. A., Ludermir, T.B., Aquino, R., Lira, M.M. and Neto, O.N. (2008). Investigating the use of reservoir computing for forecasting the hourly wind speed in short-term. In International Joint Conference on Neural Networks - IJCNN, pages 1950–1957, Hong Kong.
Ferreira, A. A. e Ludermir, T.B. (2009). Genetic algorithm for reservoir computing optimization. Neural Networks. IJCNN - International Joint Conference on, vol., no., pp.811-815.
Ferreira, A. A., Ludermir, T.B. and Aquino, R. (2013). An approach to reservoir computing design and training. Expert Systems with Applications, Volume 40, Issue 10, pp. 4172-4182.
Ferreiro, S., Arnaiz, A., Sierra, B. and Irigoien, I. (2012). Application of Bayesian networks in prognostics for a new Integrated Vehicle Health Management concept. Expert Systems with Applications, Volume 39, Issue 7, Pages 6402-6418.
Frederick, D., De Castro, J. and Litt, J. (2007). “User’s Guide for the Commercial Modular Aero-Propulsion System Simulation (CMAPSS).” NASA/ARL.
Heng, A., Zhang, S. Tan, A. and Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, Volume 23, Issue 3, pp. 724-739.
Hu, C., Youn, B.D. and Wang, P. (2011). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Prognostics and Health Management (PHM), IEEE Conference on , vol., no., pp.1-10.
Hossain, M.S. and El-Shafie, A. (2014). Evolutionary techniques versus swarm intelligences: application in reservoir release optimization. Neural Computing and Applications, volume 24, number 7-8, pp.1583-1594.
Gasperin, M., Juricic, Baskoski, P. and Jozef, V. (2011). Model-based prognostics of gear health using stochastic dynamic models. Mechanical Systems and Signal Processing 25, pp. 537–538.
Ishii, K., van der Zant, T., Becanovic, V., & Ploger, P. (2004). Identification of motion with echo state network. In Proceedings of the OCEANS MTS/IEEE–TECHNOOCEAN conference, Vol. 3, pp. 1205–1210.
Jaeger, H (2001). The Echo state approach to analyzing and training recurrent neural networks. Technical Report GMD Report 148, German National Research Center for Information Technology, URL www.faculty.jacobs-university.de/hjaeger/pubs/ EchoStatesTechRep.pdf.
Jaeger, H (2002a). Short-term memory in echo state networks. Technical report, GDM 152, German National Resource Center for Information Technology.
Jaeger, H (2002b). A tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the echo state network approach. Technical Report GMD Report 159, German National Research Center for Information Technology, URL www.faculty.jacobs-university.de/hjaeger/pubs/ESNTutorial Rev.pdf.
Jaeger, H. (2003). Adaptive nonlinear system identification with echo state networks, in Advances in Neural Information Processing Systems, S. Becker, S. Thrun, K. Obermayer, Eds., Cambridge, MA: MIT Press, pp.593-600.
Jaeger, H. and Haas, H. (2004). Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 304(5667): pp. 78-80, URL www.faculty.jacobs-university.de/ hjaeger/pubs/ ESNScience04.pdf.
Karaboga, D. and Basturk, B. (2007). A powerful and Efficient Algorithm for Numerical Function Optimization: Artificial Bee Colony (ABC) Algorithm. Journal of Global Optimization, Volume: 39, Issue: 3, pp. 459-171.
Karaboga, D. and Akay, B. (2009). A comparative study of Artificial Bee Colony algorithm. Applied Mathematics and Computation, Volume 214, Issue 1, Pages 108-132.
Karaboga, D. and Ozturk, C. (2009). Neural Networks Training by Artificial Bee Colony Algorithm on Pattern Classification. Neural Network World, 19 (3), 279-292.
Karaboga, D. and Ozturk, C. (2010). Fuzzy clustering with artificial bee colony algorithm. Scientific Research and Essays, Volume: 5 Issue: 14 Pages: 1899-1902.
Karaboga, D. (2010). Artificial bee colony algorithm. Scholarpedia, 5(3):6915.
Kumar, S., Torres, M., Chan, Y.C. and Pecht, M. (2008). A hybrid prognostics methodology for electronic products. Neural Networks. IJCNN. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, vol., no., pp. 3479-3485.
Li, D., Wang, W., and Ismail, F. (2013). Enhanced fuzzy-filtered neural networks for material fatigue prognosis. Applied Soft Computing, Vol. 13, No. 1, pp. 283–291.
Liao, L. and Kottig, F. (2014). Review of Hybrid Prognostics Approaches for Remaining Useful Life Prediction of Engineered Systems, and an Application to Battery Life Prediction. Reliability, IEEE Transactions on, vol.63, no.1, pp. 191-207.
Liu, K., Gebraeel, N. Z. and Shi, J. (2013). A data-level fusion model for developing composite health indices for degradation modeling and prognostic analysis. IEEE Trans. on Automation Science and Engineering.
Lukosevicius, M. and Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3, pp.127–149.
Maass, W., Natschlager, T. and Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531–2560.
Natschlager, T., Maass, W. and Markram, H. (2002). The ‘liquid computer’: A novel strategy for real-time computing on time series. Special Issue on Foundations of Information Processing of TELEMATIK, 8(1):39–43.
Pecht, M. (2008). Prognostics and Health Management of Electronics. John Wiley, New Jersey.
Pecht, M. and Jaai, R. (2010). A prognostics and health management roadmap for information and electronics-rich system. Microelectronics Reliability 50, pp. 317–323.
Peng, Y., Wang, H., Wang, J., Liu, D. and Peng, X. (2012). A modified echo state network based remaining useful life estimation approach. Prognostics and Health Management (PHM), IEEE Conference on, vol., no., pp.1-7.
Pla, A., López, B., Gay, P. and Pous, C. (2013). eXiT*CBR.v2: Distributed case-based reasoning tool for medical prognosis, Decision Support Systems, Volume 54, Issue 3, pp. 1499-1510.
Prechelt. L. (1994). Proben1 - a set of neural network benchmark problems and benchmarking rules. Technical report, 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany.
Ramasso, E. and Saxena, A. (2014). Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets,” International Journal of Prognostics and Health Management, v. 5, n. 2, p. 1–15.
Saxena, A. and Goebel, K. (2008). Turbofan Engine Degradation Simulation Data Set, NASA Ames Prognostics Data Repository (http://ti.arc.nasa.gov/ tech/dash/pcoe/prognostic-data-repository), NASA Ames Research Center, Moffett Field, CA.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S. and Schwabacher, M. (2008a). Metrics for Evaluating Performance of Prognositc Techniques. International Conference on Prognostics and Health Management, Denver, CO.
Saxena, A., Goebel, K., Simon, D. and Eklund, N. (2008b). Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, p. 1–9. International Conference On Prognostics And Health Management, Denver, CO, Oct 06-09.
Saxena, A., Celaya, J., Saha, B., Saha, S. and Goebel, K. (2010). Metrics for offline evaluation of prognostic performance. International Journal of Prognostics and Health Management.
Schrauwen, B., Verstraeten, D. and Campenhout, J.V. (2007). An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th European Symposium on Artificial Neural Networks, pp. 471–482.
Shankar, B. (2015). Remaining Useful Life Prediction through Failure Probability Computation for Condition-based Prognostics. In Proceedings of the Annual Conference of the Prognostics and Health Management Society.
Song, Q., Zhao, X. Feng, Z. An, Y. and Song, B. (2011). Hourly electric load forecasting algorithm based on echo state neural network, Control and Decision Conference (CCDC), 2011 Chinese, vol., no., pp.3893, 3897, 23-25 May.
Steil, J.J. (2004). Backpropagation-decorrelation: online recurrent learning with O(N) complexity. In Proc. IJCNN.
ToolboxABC (2015). MATLAB code v2 of the basic ABC algorithm. Available in: < http://mf.erciyes.edu.tr/ abc>. Access in: march 2015.
ToolboxESN (2015). Simple and very simple Matlab toolbox for Echo State Networks. Available in:
Turanoglu, E., Ozceylan, E. and Kiran, M.S. (2011). Particle Swarm Optimization and Artificial Bee Colony Approaches to Optimize of Single Input-Output Fuzzy Membership Functions. Proceedings of the 41st International Conference on Computers & Industrial Engineering.
Vachtsevanos, G., Lewis, F. L., Roemer, M., Hess, A. and Wu, B. (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems. 1st ed. Hoboken, New Jersey: John Wiley & Sons, Inc.
Verstraeten, D., Schrauwen, B., D’Haene, M. and Stroobandt, D. (2007). An experimental unification of reservoir computing methods. Neural Networks, 20:391–403.
Wang, T., (2010). Trajectory Similarity Based Prediction for Remaining Useful Life Estimation. Doctoral dissertation. University of Cincinnati.
Wang, W., Zhang, W., (2008). An asset residual life prediction model based on expert judgments. European Journal of Operational Research 188, pp. 496–505.
Weiming, W., Bing, L., Min, L. and Houjun, W. (2014). Prognostics of Lithium-Ion Batteries Based on the Verhulst Model, Particle Swarm Optimization and Particle Filter. Instrumentation and Measurement, IEEE Transactions on, vol.63, no.1, pp.2-17.
Zhang, Y., Zhao, X., Liu, W., Zhang, J., Jia, Y. and Feng, T. (2011). Research on gearbox wearing prognosis based on Gamma-State Space Model. Reliability, Maintainability and Safety (ICRMS), 9th International Conference on, vol., no., pp. 279-283.
Section
Technical Papers