TrajecNets: Online Failure Evolution Analysis in 2D Space
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Dec 2, 2019
Nauman Shahid
Anarta Ghosh
Abstract
We propose a novel Recurrent Neural Network (RNN) based autoencoder for embedding the run-to-failure time series sensor data in a 2D feature space. The embedding, extracted from the network, is in the form of a smooth trajectory, which represents the temporal evolution of data from healthy to failure states, hence the name TrajecNets. The visualizable 2D trajectory can be used directly for highly intuitive and interpretable health monitoring, which can in turn be used for Remaining Useful Life (RUL) estimation task, without compromising the performance. We also propose a novel unsupervised failure prediction methodology which uses the 2D trajectories and health curve of the time series to compute evolving failure mode probabilities. Together, the visualizable 2D trajectories and the interpretable failure mode probabilities, health curve and RUL are envisaged to provide system and maintenance engineers, insight into failure dynamics. Experiments on NASA CMAPSS Turbofan benchmark dataset show
promising results on degradation tracking, health monitoring, failure prediction and RUL estimation tasks.
##plugins.themes.bootstrap3.article.details##
Keywords
Remaining Useful Life Estimation, Predictive Maintenance, Prognostics, Machine learning, Deep learning, Recurrent Neural Networks, Aeronautics, Case Study, failure evolution
References
Aizpurua, J. I., & Catterson, V. M. (2015). Towards a methodology for design of prognostic systems. In Annual conference of the prognostics and health management society 2015 (pp. 504–517).
Akintayo, A., Lore, K. G., Sarkar, S., & Sarkar, S. (2016). Early detection of combustion instabilities using deep convolutional selective autoencoders on hi-speed flame video. arXiv preprint arXiv:1603.07839.
Cadini, F., Zio, E., & Avram, D. (2009). Model-based monte carlo state estimation for condition-based component replacement. Reliability Engineering & System Safety, 94(3), 752–758.
Chollet, F., et al. (2015). Keras. https://keras.io.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2015). Gated feedback recurrent neural networks. In International conference on machine learning (pp. 2067–2075).
Elattar, H. M., Elminir, H. K., & Riad, A. (2016). Prognostics: a literature review. Complex&Intelligent Systems, 2(2), 125–154.
Ellefsen, A. L., Bjørlykhaug, E., Æsøy, V., Ushakov, S., & Zhang, H. (2019). Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture. Reliability Engineering & System Safety, 183, 240–251.
Gugulothu, N., TV, V., Malhotra, P., Vig, L., Agarwal, P., & Shroff, G. (2017). Predicting remaining useful life using time series embeddings based on recurrent neural networks. arXiv preprint arXiv:1709.01073.
He, K., Zhang, X., Ren, S., & Sun, J. (2016, June). Deep residual learning for image recognition. In 2016 ieee conference on computer vision and pattern recognition (cvpr) (p. 770-778). doi: 10.1109/CVPR.2016.90
Hermans, M., & Schrauwen, B. (2013). Training and analysing deep recurrent neural networks. In Advances in neural information processing systems (pp. 190–198).
Ilin, A., Prémont-Schwarz, I., Hao, T. H., Rasmus, A., Boney, R., & Valpola, H. (2017). Recurrent ladder networks. arXiv preprint arXiv:1707.09219, 3(6).
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436.
Li, J., Li, X., & He, D. (2019). A directed acyclic graph network combined with cnn and lstm for remaining useful life prediction. IEEE Access, 7, 75464-75475. doi: 10.1109/ACCESS.2019.2919566
Maaten, L. v. d., & Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(Nov), 2579–2605.
Malhotra, P., TV, V., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. (2016). Multisensor prognostics using an unsupervised health index
based on lstm encoder-decoder. arXiv preprint arXiv:1608.06154.
Mosallam, A., Medjaher, K., & Zerhouni, N. (2015). Component based data-driven prognostics for complex systems: Methodology and applications. In 2015 first international conference on reliability systems engineering (icrse) (pp. 1–7).
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048.
My¨otyri, E., Pulkkinen, U., & Simola, K. (2006). Application of stochastic filtering for lifetime prediction. Reliability Engineering & System Safety, 91(2), 200–208.
Phillips, P. A. (2012). Health monitoring of electrical actuators for landing gears (Unpublished doctoral dissertation). The University of Manchester (United Kingdom).
Qiu, H., Lee, J., Lin, J., & Yu, G. (2003). Robust performance degradation assessment methods for enhanced rolling element bearing prognostics. Advanced Engineering Informatics, 17(3-4), 127–140.
Ramasso, E. (2014). Investigating computational geometry for failure prognostics. International Journal of Prognostics and Health Management, 5(1), 005.
Reddy, K. K., Venugopalan, V., & Giering, M. J. (2016). Applying deep learning for prognostic health monitoring of aerospace and building systems. In 1st acm sigkdd workshop on ml for phm.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostic techniques. In 2008 international conference on prognostics and health management (pp. 1–17).
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine runto-failure simulation. In 2008 international conference on prognostics and health management (pp. 1–9).
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first aaai conference on artificial intelligence.
Vachtsevanos, G., & Wang, P. (2001). Fault prognosis using dynamic wavelet neural networks. In 2001 ieee autotestcon proceedings. ieee systems readiness technology conference.(cat. no. 01ch37237) (pp. 857–870).
Wang, J., Wen, G., Yang, S., & Liu, Y. (2018). Remaining useful life estimation in prognostics using deep bidirectional lstm neural network. In 2018 prognostics and system health management conference (phmchongqing) (pp. 1037–1042).
Wang, T. (2010). Trajectory similarity based prediction for remaining useful life estimation (Unpublished doctoral dissertation). University of Cincinnati.
Wang, T., Yu, J., Siegel, D., & Lee, J. (2008). A similaritybased prognostics approach for remaining useful life estimation of engineered systems. In 2008 international conference on prognostics and health management (pp. 1–6).
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla lstm neural networks. Neurocomputing, 275, 167–179.
Zhang, A.,Wang, H., Li, S., Cui, Y., Liu, Z., Yang, G., & Hu, J. (2018). Transfer learning with deep recurrent neural networks for remaining useful life estimation. Applied Sciences, 8(12), 2416.
Zhang, C., Pin, L., K. Qin, A., & Chen Tan, K. (2016, 07). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE Transactions on Neural Networks and Learning Systems, PP, 1-13. doi: 10.1109/TNNLS.2016.2582798
Zhao, R., Wang, J., Yan, R., & Mao, K. (2016). Machine health monitoring with lstm networks. In 2016 10th international conference on sensing technology (icst) (pp. 1–6).
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237.
Zhao, R., Yan, R., Wang, J., & Mao, K. (2017). Learning to monitor machine health with convolutional bidirectional lstm networks. Sensors, 17(2), 273.
Akintayo, A., Lore, K. G., Sarkar, S., & Sarkar, S. (2016). Early detection of combustion instabilities using deep convolutional selective autoencoders on hi-speed flame video. arXiv preprint arXiv:1603.07839.
Cadini, F., Zio, E., & Avram, D. (2009). Model-based monte carlo state estimation for condition-based component replacement. Reliability Engineering & System Safety, 94(3), 752–758.
Chollet, F., et al. (2015). Keras. https://keras.io.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2015). Gated feedback recurrent neural networks. In International conference on machine learning (pp. 2067–2075).
Elattar, H. M., Elminir, H. K., & Riad, A. (2016). Prognostics: a literature review. Complex&Intelligent Systems, 2(2), 125–154.
Ellefsen, A. L., Bjørlykhaug, E., Æsøy, V., Ushakov, S., & Zhang, H. (2019). Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture. Reliability Engineering & System Safety, 183, 240–251.
Gugulothu, N., TV, V., Malhotra, P., Vig, L., Agarwal, P., & Shroff, G. (2017). Predicting remaining useful life using time series embeddings based on recurrent neural networks. arXiv preprint arXiv:1709.01073.
He, K., Zhang, X., Ren, S., & Sun, J. (2016, June). Deep residual learning for image recognition. In 2016 ieee conference on computer vision and pattern recognition (cvpr) (p. 770-778). doi: 10.1109/CVPR.2016.90
Hermans, M., & Schrauwen, B. (2013). Training and analysing deep recurrent neural networks. In Advances in neural information processing systems (pp. 190–198).
Ilin, A., Prémont-Schwarz, I., Hao, T. H., Rasmus, A., Boney, R., & Valpola, H. (2017). Recurrent ladder networks. arXiv preprint arXiv:1707.09219, 3(6).
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436.
Li, J., Li, X., & He, D. (2019). A directed acyclic graph network combined with cnn and lstm for remaining useful life prediction. IEEE Access, 7, 75464-75475. doi: 10.1109/ACCESS.2019.2919566
Maaten, L. v. d., & Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(Nov), 2579–2605.
Malhotra, P., TV, V., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. (2016). Multisensor prognostics using an unsupervised health index
based on lstm encoder-decoder. arXiv preprint arXiv:1608.06154.
Mosallam, A., Medjaher, K., & Zerhouni, N. (2015). Component based data-driven prognostics for complex systems: Methodology and applications. In 2015 first international conference on reliability systems engineering (icrse) (pp. 1–7).
Mosallam, A., Medjaher, K., & Zerhouni, N. (2016). Data-driven prognostic method based on bayesian approaches for direct remaining useful life prediction. Journal of Intelligent Manufacturing, 27(5), 1037–1048.
My¨otyri, E., Pulkkinen, U., & Simola, K. (2006). Application of stochastic filtering for lifetime prediction. Reliability Engineering & System Safety, 91(2), 200–208.
Phillips, P. A. (2012). Health monitoring of electrical actuators for landing gears (Unpublished doctoral dissertation). The University of Manchester (United Kingdom).
Qiu, H., Lee, J., Lin, J., & Yu, G. (2003). Robust performance degradation assessment methods for enhanced rolling element bearing prognostics. Advanced Engineering Informatics, 17(3-4), 127–140.
Ramasso, E. (2014). Investigating computational geometry for failure prognostics. International Journal of Prognostics and Health Management, 5(1), 005.
Reddy, K. K., Venugopalan, V., & Giering, M. J. (2016). Applying deep learning for prognostic health monitoring of aerospace and building systems. In 1st acm sigkdd workshop on ml for phm.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostic techniques. In 2008 international conference on prognostics and health management (pp. 1–17).
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine runto-failure simulation. In 2008 international conference on prognostics and health management (pp. 1–9).
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first aaai conference on artificial intelligence.
Vachtsevanos, G., & Wang, P. (2001). Fault prognosis using dynamic wavelet neural networks. In 2001 ieee autotestcon proceedings. ieee systems readiness technology conference.(cat. no. 01ch37237) (pp. 857–870).
Wang, J., Wen, G., Yang, S., & Liu, Y. (2018). Remaining useful life estimation in prognostics using deep bidirectional lstm neural network. In 2018 prognostics and system health management conference (phmchongqing) (pp. 1037–1042).
Wang, T. (2010). Trajectory similarity based prediction for remaining useful life estimation (Unpublished doctoral dissertation). University of Cincinnati.
Wang, T., Yu, J., Siegel, D., & Lee, J. (2008). A similaritybased prognostics approach for remaining useful life estimation of engineered systems. In 2008 international conference on prognostics and health management (pp. 1–6).
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla lstm neural networks. Neurocomputing, 275, 167–179.
Zhang, A.,Wang, H., Li, S., Cui, Y., Liu, Z., Yang, G., & Hu, J. (2018). Transfer learning with deep recurrent neural networks for remaining useful life estimation. Applied Sciences, 8(12), 2416.
Zhang, C., Pin, L., K. Qin, A., & Chen Tan, K. (2016, 07). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE Transactions on Neural Networks and Learning Systems, PP, 1-13. doi: 10.1109/TNNLS.2016.2582798
Zhao, R., Wang, J., Yan, R., & Mao, K. (2016). Machine health monitoring with lstm networks. In 2016 10th international conference on sensing technology (icst) (pp. 1–6).
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237.
Zhao, R., Yan, R., Wang, J., & Mao, K. (2017). Learning to monitor machine health with convolutional bidirectional lstm networks. Sensors, 17(2), 273.
Issue
Section
Technical Papers