A CNN-Multi-Head Attention Framework for Gearbox Incremental Fault Diagnosis Under Non-Stationary Conditions
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Deep learning-based gearbox fault diagnosis approaches have demonstrated exceptional performance in achieving accurate fault identification across diverse industrial applications. Nonetheless, machines frequently operate under conditions characterized by time-varying speeds or loads, known as non-stationary working conditions. When a series of different non-stationary conditions tasks are sequentially input into the model for training, an issue arises where the model tends to forget previous tasks, a phenomenon referred to as "catastrophic forgetting". To address the challenge posed by task increments within non-stationary conditions, this paper proposes an incremental learning-based multi-task fault diagnosis framework under non-stationary conditions. This methodology enhances the model's diagnostic capabilities under non-stationary conditions by amalgamating convolutional neural network (CNN) with multi-head self-attention mechanisms. It employs exemplar replay and hybrid cross-head knowledge distillation techniques to preserve the model's understanding of prior tasks, thereby facilitating the incremental learning of multiple tasks. The efficacy of this proposed framework is substantiated through its application on the MCC5-THU fault diagnosis datasets of gearbox under time-varying speed working conditions. Experimental results demonstrate that this approach significantly mitigates the "catastrophic forgetting" effect, thereby offering a robust solution for multi-tasks increment fault diagnosis of gearbox operating under non-stationary conditions.
##plugins.themes.bootstrap3.article.details##
Gearbox fault diagnostics, Incremental learning, Non-stationary conditions, Knowledge distillation
Peng, Y., Qiao, W., Cheng, F., & Qu, L. (2021). Wind turbine drivetrain gearbox fault diagnosis using information fusion on vibration and current signals. IEEE Transactions on Instrumentation and Measurement, 70, 1-11.
Feng, Z., Chen, X., & Zuo, M. J. (2018). Induction motor stator current AM-FM model and demodulation analysis for planetary gearbox fault diagnosis. IEEE Transactions on industrial informatics, 15(4), 2386-2394.
Yin, Z., & Hou, J. (2016). Recent advances on SVM based fault diagnosis and process monitoring in complicated industrial processes. Neurocomputing, 174, 643-650.
Sinitsin, V., Ibryaeva, O., Sakovskaya, V., & Eremeeva, V. (2022). Intelligent bearing fault diagnosis method combining mixed input and hybrid CNN-MLP model. Mechanical Systems and Signal Processing, 180, 109454.
Qin, G., Zhang, K., Lai, X., Zheng, Q., Ding, G., Zhao, M., & Zhang, Y. (2024). An adaptive symmetric loss in dynamic wide-kernel ResNet for rotating machinery fault diagnosis under noisy labels. IEEE Transactions on Instrumentation and Measurement, 73, 1-12.
Lin, X., Li, B., Yang, X., & Wang, J. (2018, December). Fault diagnosis of aero-engine bearing using a stacked auto-encoder network. In 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC) (pp. 545-548). IEEE.
Yu, G., Wu, P., Lv, Z., Hou, J., Ma, B., & Han, Y. (2023). Few-shot fault diagnosis method of rotating machinery using novel MCGM based CNN. IEEE Transactions on Industrial Informatics, 19(11), 10944-10955.
Liu, R., Wang, F., Yang, B., & Qin, S. J. (2019). Multiscale kernel based residual convolutional neural network for motor fault diagnosis under nonstationary conditions. IEEE Transactions on Industrial Informatics, 16(6), 3797-3806.
Dong, Y., Jiang, H., Yao, R., Mu, M., & Yang, Q. (2024). Rolling bearing intelligent fault diagnosis towards variable speed and imbalanced samples using multiscale dynamic supervised contrast learning. Reliability Engineering & System Safety, 243, 109805.
Zhao, M., Kang, M., Tang, B., & Pecht, M. (2017). Deep residual networks with dynamically weighted wavelet coefficients for fault diagnosis of planetary gearboxes. IEEE Transactions on Industrial Electronics, 65(5), 4290-4300.
Zhao, X., Yao, J., Deng, W., Ding, P., Ding, Y., Jia, M., & Liu, Z. (2022). Intelligent fault diagnosis of gearbox under variable working conditions with adaptive intraclass and interclass convolutional neural network. IEEE Transactions on Neural Networks and Learning Systems, 34(9), 6339-6353.
Wang, P., Xiong, H., & He, H. (2023). Bearing fault diagnosis under various conditions using an incremental learning-based multi-task shared classifier. Knowledge-based systems, 266, 110395.
Shi, M., Ding, C., Chang, S., Shen, C., Huang, W., & Zhu, Z. (2024). Cross-domain class incremental broad network for continuous diagnosis of rotating machinery faults under variable operating conditions. IEEE Transactions on Industrial Informatics, 20(4), 6356-6368.
Wang, L., Liu, S., & Xiao, H. (2024). Vaccine enhanced continual learning with TFE to overcome catastrophic forgetting for variable speed-bearing fault diagnosis. IEEE Transactions on Industrial Informatics, 20(5), 7112-7123.
Ostapenko, O., Puscas, M., Klein, T., Jahnichen, P., & Nabi, M. (2019). Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11321-11329).
Rebuffi, S. A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017). icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 2001-2010).
Yin, H., Molchanov, P., Alvarez, J. M., Li, Z., Mallya, A., Hoiem, D., ... & Kautz, J. (2020). Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8715-8724).
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526.
Li, Z., & Hoiem, D. (2017). Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12), 2935-2947.
Lopez-Paz, D., & Ranzato, M. A. (2017). Gradient episodic memory for continual learning. Advances in neural information processing systems, 30.
Mallya, A., & Lazebnik, S. (2018). Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 7765-7773).
Veniat, T., Denoyer, L., & Ranzato, M. A. (2020). Efficient continual learning with modular networks and task-driven priors. arXiv preprint arXiv:2012.12631.
Zhang, Y., Shen, C., Shi, J., Li, C., Lin, X., Zhu, Z., & Wang, D. (2024). Deep adaptive sparse residual networks: A lifelong learning framework for rotating machinery fault diagnosis with domain increments. Knowledge-Based Systems, 293, 111679.
Chen, B., Shen, C., Wang, D., Kong, L., Chen, L., & Zhu, Z. (2022). A lifelong learning method for gearbox diagnosis with incremental fault types. IEEE transactions on instrumentation and measurement, 71, 1-10.
Chen, B., Shen, C., Li, L., Shi, J., Huang, W., & Zhu, Z. (2023, October). Continual unsupervised domain adaptation for bearing fault diagnosis under variable working conditions. In International Conference on Electrical and Information Technologies for Rail Transportation (pp. 395-403). Singapore: Springer Nature Singapore.
Wang, J., Chen, Y., Zheng, Z., Li, X., Cheng, M. M., & Hou, Q. (2024). CrossKD: Cross-head knowledge distillation for object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16520-16530).
Chen, S., Liu, Z., He, X., Zou, D., & Zhou, D. (2024). Multi-mode fault diagnosis datasets of gearbox under variable working conditions. Data in brief, 54, 110453.
This work is licensed under a Creative Commons Attribution 3.0 Unported License.