Ensemble Deep Learning for Detecting Onset of Abnormal Operation in Industrial Multi-component Systems
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Oct 24, 2022
Balaji Selvanathan
Sri Harsha Nistala
Venkataramana Runkana
Saurabh Jaywant Desai
Shashank Agarwal
Abstract
Breakdowns and unplanned shutdowns in industrial processes and equipment can lead to significant loss of availability and revenue. It is imperative to perform optimal maintenance of such systems when signs of abnormal behavior are detected and before they propagate and lead to catastrophic failure. This is particularly challenging in systems with interconnected multiple components as it is difficult to isolate the effect of one component on the operation of other components in the system. In this work, an ensemble approach based on Cascaded Convolutional neural network and Long Short-term Memory (CC-LSTM) network models is proposed for detecting and predicting the time of onset of faults in interconnected multicomponent systems. The performance of the ensemble CC-LSTM model was demonstrated on an industrial 4-component system and was found to improve the accuracy of onset time predictions by ~15% compared to individual CC-LSTM models and ~25-40% compared to commonly used deep learning techniques such as dense neural networks, convolutional neural networks and LSTMs. The CC-LSTM and the ensemble models also had the lowest missed detection rates and zero false positive rates making them ideal for real-time monitoring and fault detection in multicomponent systems.
##plugins.themes.bootstrap3.article.details##
Keywords
Industrial, Multi-component systems, Abnormal Operation Onset Detection, Deep Learning, Ensemble
References
Aljbali, S., & Roy, K. (2021). Anomaly Detection Using Bidirectional LSTM. Advances in Intelligent Systems and Computing (pp. 612-619). Amsterdam: Springer.
Altarabichi, M., Mashhadi, P., Fan, Y., Pashami, S., Nowaczyk, S., Moral, P., . . . Rognvaldsson, T. (2020). Stacking ensembles of heterogeneous classifiers for fault detection in evolving environments. The 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference (p. 1068). Venice, Italy: Research Publishing.
Assaf, R., Do, P., Scarf, P., & Nefti-Meziani, S. (2017). Diagnosis for Systems with Multi-component Wear Interactions. IEEE Intl Conf on Prognostics and Health Management. Dallas, TX, USA: IEEE.
Bansal, M. (2010). Performance evaluation of butterworth filter for signal denoising. International Journal of Electronics and Communication Technology, 59-62.
Bian, L., & Gebraeel, N. (2014). Stochastic modeling and real-time prognostics for multi-component systems with degradation rate interactions. IIE Transactions, 470-482.
Canizo, M., Triguero, I., Conde, A., & Onieva, E. (2019). Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing, 246-260.
Cannarile, M., Compare, M., Bareldi, P., Yang, Z., & Zio, E. (2020, August). The Aramis Challenge: Prognostics and Health Management in Evolving Environments. Retrieved from ESREL2020-PSAM15: https://www.esrel2020-psam15.org/Aramis_Challenge.pdf
Chalapathy, R., & Chawla, S. (2019). Deep learning for anomaly detection: A survey.
Eren, L., Ince, T., & Kiranyaz, S. (2019). A generic intelligent bearing fault diganosis system using compact adpative 1D CNN classifier. Journal of Signal Processing Systems, 179-189.
Gupta, A., Masampally, V. S., Jadhav, V., Deodhar, A., & Runkana, V. (2021). Supervised Operational Change Point Detection using Ensemble Long-Short Term Memory in a Multicomponent Industrial System. IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI). Herl'any, Slovakia: IEEE.
Hasan, M., Sohaib, M., & Kim, J. (2019). 1D CNN based transfer learning based model for bearing fault diagnosis under variable working conditions. Advances in Intelligent Systems and Computing, 888.
Introduction to the Keras Tuner. (2022, March 25). Retrieved from Tensorflow: https://www.tensorflow.org/tutorials/keras/keras_tuner
Keras API Reference. (2021, May 05). Retrieved from Keras: https://keras.io/api/
Kim, T., & Cho, S. (2018). Web traffic anomaly detection using C-LSTM neural networks. Expert Systems with Applications, 66-76.
Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, D. (2021). 1D convolutional neural networks and applications: A survey. Mechanical Systems and Signal Processing, 107398.
Krizhevsky, A., Sutskever, I., & Hinton, G. (2017). Imagenet classification with deep convolutional neural networks. Communication of the ACM, 84-90.
Lai, Y., Zhang, J., & Liu, Z. (2019). Industry anomaly detection and attack classification method based on convolutional neural network. Security and Communication Networks.
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2018). Hyperband: A novel banditbased approach for hyperparameter optimization. Journal of Machine Learning Research, 1-52.
Li, Y., Zou, L., Jiang, L., & Zhou, X. (2019). Fault diagnosis of rotating machinery based on combination of deep belief network and one dimensional convolutional neural network. IEEE Access, 7.
Li, Z., Li, J., Wang, Y., & Wang, K. (2019). A deep learning approach for anomaly detection based on SAE and LSTM in mechanical equipment. International Journal of Advanced Manufacturing Technology, 499-510.
Lin, Y., Zakwan, S., & Jennions, I. (2020). A Bayesian Approach to Fault Identification in the Presence of Multi-component Degradation. International Journal of Prognostics and Health Management, 1-9.
Lindemann, B., Jazdi, N., & Weyrich, M. (2020). Anomaly detection and prediction in discrete manufacturing based on cooperative LSTM networks. IEEE 16th International Conference on Automation Science and Engineering (CASE), (pp. 1003-1010). Hong Kong.
Lindemann, B., Maschler, B., Sahlab, N., & Weyrich, M. (2021). A survey on anomaly detection for technical systems using LSTM networks. Computers in Industry, 103498.
Liu, H., Zhao, Y., Zaporowska, A., & Zakwan, S. (2020). A machine learning-based clustering approach to diagnose multi-component degradation of aircraft fuel systems. Neural Computing and Applications.
Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. (2016). LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection. ICML Anomaly Detection Workshop. New York, NY, USA.
Prakash, O., Samantaray, A., Bhattacharyya, R., & Ghoshal, S. (2018). Adaptive Prognosis for a Multicomponent Dynamical System of Unknown Degradation Modes. IFAC PapersOnLine, 184-191.
Rocchetta, R., Petkovic, M., & Gao, Q. (2020). Scenariobased Generalization bound for Anomaly Detection Support Vector Machine Ensembles. Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference, (pp. 1069-1076). Venice, Italy
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostics techniques. International Conference on Prognostics and Health Management
Shang, C., Yang, F., Huang, D., & Lyu, W. (2014). A multiscale feature learning scheme based on deep learning for industrial process monitoring and fault diagnosis. Journal of Process Control, 223-233.
Siahpour, S., Ainapure, A., Li, X., & Lee, J. (2020). A Deep Learning Framework for Health Anomaly Detection of Multi-component Systems in Evolving Environments: A case study in PHM. 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conference (pp. 1077-1084). Venice, Italy: Research Publishing.
Snedecor, G., & Cochran, W. (1989). Statistical Methods. Iowa State University Press.
Ullah, W., Ullah, A., Haq, I., Muhammad, K., Sajjad, M., & Baik, S. (2020). CNN features with bi-directional LSTM real-time anomaly detection in surveillance networks. Multimedia tools and applications, 80.
Wang, D., Guo, Q., Song, Y., Gao, S., & Li, Y. (2019). Application of multiscale learning neural network based on CNN in bearing fault diagnosis. Journal of Signal Processing Systems, 91.
Wang, X., Mao, D., & Li, X. (2020). Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network. Measurement.
Yang, Z., Baraldi, P., & Zio, E. (2022). A method for fault detection in multi-component systems based on sparse autoencoder-based deep neural networks. Reliability Engineering and System Safety, 220, 1-15.
Yuan, J., & Tian, Y. (2019). A multiscale feature learning scheme based on deep learning for industrial process monitoring and fault diagnosis. IEEE Access, 151189-151202.
Zhang, W., Yang, D., & Wang, H. (2019). Data-Driven Methods for Predictive Maintenance of Industrial Equipment: A Survey. IEEE Systems Journal, 2213-2227.
Zhao, G., Zhang, G., Ge, Q., & Liu, X. (2016). Research advances in fault diagnosis and prognostics based on deep learning. Prognostics and System Health Management Conference, (pp. 1-6). Chengdu.
Zheng, L., Xue, W., Chen, F., Guo, P., Chen, J., Chen, B., & Gao, H. (2019). A fault prediction of equipment based on CNN-LSTM network. IEEE International Conference on Energy Internet, (pp. 537-541).
Zhou, C., Sun, C., Liu, Z., & Lau, F. (2015). A C-LSTM Neural Network for Text Classification. arXiv, arXiv:1511.08630.
Altarabichi, M., Mashhadi, P., Fan, Y., Pashami, S., Nowaczyk, S., Moral, P., . . . Rognvaldsson, T. (2020). Stacking ensembles of heterogeneous classifiers for fault detection in evolving environments. The 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference (p. 1068). Venice, Italy: Research Publishing.
Assaf, R., Do, P., Scarf, P., & Nefti-Meziani, S. (2017). Diagnosis for Systems with Multi-component Wear Interactions. IEEE Intl Conf on Prognostics and Health Management. Dallas, TX, USA: IEEE.
Bansal, M. (2010). Performance evaluation of butterworth filter for signal denoising. International Journal of Electronics and Communication Technology, 59-62.
Bian, L., & Gebraeel, N. (2014). Stochastic modeling and real-time prognostics for multi-component systems with degradation rate interactions. IIE Transactions, 470-482.
Canizo, M., Triguero, I., Conde, A., & Onieva, E. (2019). Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing, 246-260.
Cannarile, M., Compare, M., Bareldi, P., Yang, Z., & Zio, E. (2020, August). The Aramis Challenge: Prognostics and Health Management in Evolving Environments. Retrieved from ESREL2020-PSAM15: https://www.esrel2020-psam15.org/Aramis_Challenge.pdf
Chalapathy, R., & Chawla, S. (2019). Deep learning for anomaly detection: A survey.
Eren, L., Ince, T., & Kiranyaz, S. (2019). A generic intelligent bearing fault diganosis system using compact adpative 1D CNN classifier. Journal of Signal Processing Systems, 179-189.
Gupta, A., Masampally, V. S., Jadhav, V., Deodhar, A., & Runkana, V. (2021). Supervised Operational Change Point Detection using Ensemble Long-Short Term Memory in a Multicomponent Industrial System. IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI). Herl'any, Slovakia: IEEE.
Hasan, M., Sohaib, M., & Kim, J. (2019). 1D CNN based transfer learning based model for bearing fault diagnosis under variable working conditions. Advances in Intelligent Systems and Computing, 888.
Introduction to the Keras Tuner. (2022, March 25). Retrieved from Tensorflow: https://www.tensorflow.org/tutorials/keras/keras_tuner
Keras API Reference. (2021, May 05). Retrieved from Keras: https://keras.io/api/
Kim, T., & Cho, S. (2018). Web traffic anomaly detection using C-LSTM neural networks. Expert Systems with Applications, 66-76.
Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, D. (2021). 1D convolutional neural networks and applications: A survey. Mechanical Systems and Signal Processing, 107398.
Krizhevsky, A., Sutskever, I., & Hinton, G. (2017). Imagenet classification with deep convolutional neural networks. Communication of the ACM, 84-90.
Lai, Y., Zhang, J., & Liu, Z. (2019). Industry anomaly detection and attack classification method based on convolutional neural network. Security and Communication Networks.
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2018). Hyperband: A novel banditbased approach for hyperparameter optimization. Journal of Machine Learning Research, 1-52.
Li, Y., Zou, L., Jiang, L., & Zhou, X. (2019). Fault diagnosis of rotating machinery based on combination of deep belief network and one dimensional convolutional neural network. IEEE Access, 7.
Li, Z., Li, J., Wang, Y., & Wang, K. (2019). A deep learning approach for anomaly detection based on SAE and LSTM in mechanical equipment. International Journal of Advanced Manufacturing Technology, 499-510.
Lin, Y., Zakwan, S., & Jennions, I. (2020). A Bayesian Approach to Fault Identification in the Presence of Multi-component Degradation. International Journal of Prognostics and Health Management, 1-9.
Lindemann, B., Jazdi, N., & Weyrich, M. (2020). Anomaly detection and prediction in discrete manufacturing based on cooperative LSTM networks. IEEE 16th International Conference on Automation Science and Engineering (CASE), (pp. 1003-1010). Hong Kong.
Lindemann, B., Maschler, B., Sahlab, N., & Weyrich, M. (2021). A survey on anomaly detection for technical systems using LSTM networks. Computers in Industry, 103498.
Liu, H., Zhao, Y., Zaporowska, A., & Zakwan, S. (2020). A machine learning-based clustering approach to diagnose multi-component degradation of aircraft fuel systems. Neural Computing and Applications.
Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., & Shroff, G. (2016). LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection. ICML Anomaly Detection Workshop. New York, NY, USA.
Prakash, O., Samantaray, A., Bhattacharyya, R., & Ghoshal, S. (2018). Adaptive Prognosis for a Multicomponent Dynamical System of Unknown Degradation Modes. IFAC PapersOnLine, 184-191.
Rocchetta, R., Petkovic, M., & Gao, Q. (2020). Scenariobased Generalization bound for Anomaly Detection Support Vector Machine Ensembles. Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference, (pp. 1069-1076). Venice, Italy
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostics techniques. International Conference on Prognostics and Health Management
Shang, C., Yang, F., Huang, D., & Lyu, W. (2014). A multiscale feature learning scheme based on deep learning for industrial process monitoring and fault diagnosis. Journal of Process Control, 223-233.
Siahpour, S., Ainapure, A., Li, X., & Lee, J. (2020). A Deep Learning Framework for Health Anomaly Detection of Multi-component Systems in Evolving Environments: A case study in PHM. 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conference (pp. 1077-1084). Venice, Italy: Research Publishing.
Snedecor, G., & Cochran, W. (1989). Statistical Methods. Iowa State University Press.
Ullah, W., Ullah, A., Haq, I., Muhammad, K., Sajjad, M., & Baik, S. (2020). CNN features with bi-directional LSTM real-time anomaly detection in surveillance networks. Multimedia tools and applications, 80.
Wang, D., Guo, Q., Song, Y., Gao, S., & Li, Y. (2019). Application of multiscale learning neural network based on CNN in bearing fault diagnosis. Journal of Signal Processing Systems, 91.
Wang, X., Mao, D., & Li, X. (2020). Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network. Measurement.
Yang, Z., Baraldi, P., & Zio, E. (2022). A method for fault detection in multi-component systems based on sparse autoencoder-based deep neural networks. Reliability Engineering and System Safety, 220, 1-15.
Yuan, J., & Tian, Y. (2019). A multiscale feature learning scheme based on deep learning for industrial process monitoring and fault diagnosis. IEEE Access, 151189-151202.
Zhang, W., Yang, D., & Wang, H. (2019). Data-Driven Methods for Predictive Maintenance of Industrial Equipment: A Survey. IEEE Systems Journal, 2213-2227.
Zhao, G., Zhang, G., Ge, Q., & Liu, X. (2016). Research advances in fault diagnosis and prognostics based on deep learning. Prognostics and System Health Management Conference, (pp. 1-6). Chengdu.
Zheng, L., Xue, W., Chen, F., Guo, P., Chen, J., Chen, B., & Gao, H. (2019). A fault prediction of equipment based on CNN-LSTM network. IEEE International Conference on Energy Internet, (pp. 537-541).
Zhou, C., Sun, C., Liu, Z., & Lau, F. (2015). A C-LSTM Neural Network for Text Classification. arXiv, arXiv:1511.08630.
Section
Technical Papers