Similarity-based Multi-source Transfer Learning Approach for Time Series Classification

##plugins.themes.bootstrap3.article.main##

##plugins.themes.bootstrap3.article.sidebar##

Published Oct 17, 2022
Ayantha Senanayaka Abdullah Al Mamun Glenn Bond Wenmeng Tian Haifeng Wang Sara Fuller TC Falls Shahram Rahimi Linkan Bian

Abstract

This study aims to develop an effective method of classification concerning time series signals for machine state prediction to advance predictive maintenance (PdM).   Conventional machine learning (ML) algorithms are widely adopted in PdM, however, most existing methods assume that the training (source) and testing (target) data follow the same distribution, and that labeled data are available in both source and target domains. For real-world PdM applications, the heterogeneity in machine original equipment manufacturers (OEMs), operating conditions, facility environment, and maintenance records collectively lead to heterogeneous distribution for data collected from different machines. This will significantly limit the performance of conventional ML algorithms in PdM. Moreover, labeling data is generally costly and time-consuming. Finally, industrial processes incorporate complex conditions, and unpredictable breakdown modes lead to extreme complexities for PdM. In this study, similarity-based multi-source transfer learning (SiMuS-TL) approach is proposed for real-time classification of time series signals. A new domain, called "mixed domain," is established to model the hidden similarities among the multiple sources and the target. The proposed SiMuS-TL model mainly includes three key steps: 1) learning group-based feature patterns, 2) developing group-based pre-trained models, and 3) weight transferring. The proposed SiMuS-TL model is validated by observing the state of the rotating machinery using a dataset collected on the Skill boss manufacturing system, publicly available standard bearing datasets, Case Western Reserve University (CWRU), and Paderborn University (PU) bearing datasets. The results of the performance comparison demonstrate that the proposed SiMuS-TL method outperformed conventional Support Vector Machine (SVM), Artificial Neural Network (ANN), and Transfer learning with neural networks (TLNN) without similarity-based transfer learning methods.

Abstract 1406 | PDF Downloads 822

##plugins.themes.bootstrap3.article.details##

Keywords

Predictive maintenance, Health state prediction, Fault diagnostics and prognostics, Fault state classification, Rotating machinery, Transfer learning

References
Afridi, M. J., Ross, A., & Shapiro, E. M. (2016). L-CNN: Exploiting labeling latency in a CNN learning framework. Proceedings - International Conference on Pattern Recognition, 0, 2156–2161. https://doi.org/10.1109/ICPR.2016.7899955
AMATROL Inc. (n.d.). Skill Boss Manufacturing: MSSC CPT Plus Certification & Assessment. https://amatrol.com/product/skill-boss-performance-based-assessment/
Azeem, N., Yuan, X., Urooj, I., & Jabbar, J. (2019). Vibration-Based Power Spectral Density Analysis for the Detection of Multiple Faults in Rolling Element Bearings. 2019 5th International Conference on Control, Automation and Robotics, ICCAR 2019, April, 719–726. https://doi.org/10.1109/ICCAR.2019.8813353
Caesarendra, W., & Tjahjowidodo, T. (2017). A review of feature extraction methods in vibration-based condition monitoring and its application for degradation trend estimation of low-speed slew bearing. Machines, 5(4). https://doi.org/10.3390/machines5040021
Case Western Reserve University. (n.d.). Bearing Data Center Seeded Fault Test Data. https://engineering.case.edu/bearingdatacenter
Chen, W. H. (2012). Online fault diagnosis for power transmission networks using fuzzy digraph models. IEEE Transactions on Power Delivery, 27(2), 688–698. https://doi.org/10.1109/TPWRD.2011.2178079
De Felice, F., Petrillo, A., & Autorino, C. (2014). Maintenance strategies and innovative approaches in the pharmaceutical industry: An integrated management system (ims). International Journal of Engineering Business Management, 6(1), 1–9. https://doi.org/10.5772/59023
Donoho, D. L. (1995). De-Noising by Soft-Thresholding. 41(3).
Flandrin, P., Rilling, G., & Gonçalvés, P. (2004). Empirical mode decomposition as a filter bank. IEEE Signal Processing Letters, 11(2 PART I), 112–114. https://doi.org/10.1109/LSP.2003.821662
Gan, M., Wang, C., & Zhu, C. (2016). Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings. Mechanical Systems and Signal Processing, 72–73, 92–104. https://doi.org/10.1016/j.ymssp.2015.11.014
Gao, Z., Cecati, C., & Ding, S. X. (2015). A survey of fault diagnosis and fault-tolerant techniques-part I: Fault diagnosis with model-based and signal-based approaches. IEEE Transactions on Industrial Electronics, 62(6), 3757–3767. https://doi.org/10.1109/TIE.2015.2417501
Gideon, J., Khorram, S., Aldeneh, Z., Dimitriadis, D., & Provost, E. M. (2017). Progressive neural networks for transfer learning in emotion recognition. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2017-Augus, 1098–1102. https://doi.org/10.21437/Interspeech.2017-1637
Guo, L., Lei, Y., Xing, S., Yan, T., & Li, N. (2019). Deep Convolutional Transfer Learning Network: A New Method for Intelligent Fault Diagnosis of Machines with Unlabeled Data. IEEE Transactions on Industrial Electronics, 66(9), 7316–7325. https://doi.org/10.1109/TIE.2018.2877090
JACKSON, J. E. (1991). A User ’ s Guide to Principal Components . by J . Edward Jackson Review by : Brian D. In A Wiley-Interscience (Issue March).
Jia, F., Lei, Y., Lin, J., Zhou, X., & Lu, N. (2016). Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mechanical Systems and Signal Processing, 72–73, 303–315. https://doi.org/10.1016/j.ymssp.2015.10.025
Jin, X., Weiss, B. A., Siegel, D., & Lee, J. (2016). Present status and future growth of advanced maintenance technology and strategy in us manufacturing. International Journal of Prognostics and Health Management, 7(3). https://doi.org/10.36001/ijphm.2016.v7i3.2409
Kimotho, J. K., Lessmeier, C., Sextro, W., & Zimmer, D. (2016). Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: a benchmark data set for data-driven classification. Third European Conference of the Prognostics and Health Management Society 2016, Cm, 152–156. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1088.9087&rep=rep1&type=pdf
Li, X., Li, X., & Ma, H. (2020). Deep representation clustering-based fault diagnosis method with unsupervised data applied to rotating machinery. Mechanical Systems and Signal Processing, 143. https://doi.org/10.1016/j.ymssp.2020.106825
Li, X., & Zhang, W. (2020). Deep Learning-Based Partial Domain Adaptation Method on Intelligent Machinery Fault Diagnostics. IEEE Transactions on Industrial Electronics, 0046(c), 1–1. https://doi.org/10.1109/tie.2020.2984968
Li, X., Zhang, W., Ma, H., Luo, Z., & Li, X. (2020). Partial transfer learning in machinery cross-domain fault diagnostics using class-weighted adversarial networks. Neural Networks, 129, 313–322. https://doi.org/10.1016/j.neunet.2020.06.014
Li, Z., Wang, K., & He, Y. (2016). Industry 4.0 - Potentials for Predictive Maintenance. Iwama, 42–46. https://doi.org/10.2991/iwama-16.2016.8
Liu, Han, Zhou, J., Xu, Y., Zheng, Y., Peng, X., & Jiang, W. (2018). Unsupervised fault diagnosis of rolling bearings using a deep neural network based on generative adversarial networks. Neurocomputing, 315, 412–424. https://doi.org/10.1016/j.neucom.2018.07.034
Liu, Hongmei, Li, L., & Ma, J. (2016). Rolling Bearing Fault Diagnosis Based on STFT-Deep Learning and Sound Signals. Shock and Vibration, 2016. https://doi.org/10.1155/2016/6127479
Lu, C., Wang, Z. Y., Qin, W. L., & Ma, J. (2017). Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification. Signal Processing, 130, 377–388. https://doi.org/10.1016/j.sigpro.2016.07.028
Lu, S., He, Q., Yuan, T., & Kong, F. (2017). Online Fault Diagnosis of Motor Bearing via Stochastic-Resonance-Based Adaptive Filter in an Embedded System. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(7), 1111–1122. https://doi.org/10.1109/TSMC.2016.2531692
Macklin, C. (2019). Measuring Generalization and Overfitting in Machine Learning. UC Berkeley Electronic Theses and Dissertations, 170. https://escholarship.org/uc/item/98384265
Mao, W., He, J., Li, Y., & Yan, Y. (2016). Bearing fault diagnosis with auto-encoder extreme learning machine : A comparative study. 0(0), 1–19. https://doi.org/10.1177/0954406216675896
Mohan, J., Krishnaveni, V., & Guo, Y. (2014). A survey on the magnetic resonance image denoising methods. Biomedical Signal Processing and Control, 9(1), 56–69. https://doi.org/10.1016/j.bspc.2013.10.007
Murtagh, F., & Legendre, P. (2014). Ward’s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward’s Criterion? Journal of Classification, 31(3), 274–295. https://doi.org/10.1007/s00357-014-9161-z
Namuduri, S., Narayanan, B. N., Davuluru, V. S. P., Burton, L., & Bhansali, S. (2020). Review—Deep Learning Methods for Sensor Based Predictive Maintenance and Future Perspectives for Electrochemical Sensors. Journal of The Electrochemical Society, 167(3), 037552. https://doi.org/10.1149/1945-7111/ab67a8
Niu, S., Liu, Y., Wang, J., & Song, H. (2020). A Decade Survey of Transfer Learning (2010–2020). IEEE Transactions on Artificial Intelligence, 1(2), 151–166. https://doi.org/10.1109/TAI.2021.3054609
Niu, S., Liu, Y., Wang, J., & Song, H. (2021). A Decade Survey of Transfer Learning (2010–2020). IEEE Transactions on Artificial Intelligence, 1(2), 151–166. https://doi.org/10.1109/tai.2021.3054609
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014). Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1717–1724. https://doi.org/10.1109/CVPR.2014.222
Pan, S. J., & Yang, Q. (2009). A Survey on Transfer Learning. https://doi.org/10.1109/TKDE.2009.191
Paul, A., Rottensteiner, F., & Heipke, C. (2015). Transfer learning based on logistic regression. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, 40(3W3), 145–152. https://doi.org/10.5194/isprsarchives-XL-3-W3-145-2015
Perschl, F., & Schmidt, G. (1993). Model- and knowledge-based fault detection and diagnosis of gas transmission networks. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, 1, 749–756. https://doi.org/10.1109/icsmc.1993.384834
RDI Vibration Monitoring Equipment | Motion Amplification® & Analysis. (n.d.). https://rditechnologies.com/
Samanta, B., & Al-Balushi, K. R. (2003). Artificial neural network based fault diagnostics of rolling element bearings using time-domain features. Mechanical Systems and Signal Processing, 17(2), 317–328. https://doi.org/10.1006/mssp.2001.1462
Saruhan, H., Saridemir, S., Çiçek, A., & Uygur, I. (2014). Vibration analysis of rolling element bearings defects. Journal of Applied Research and Technology, 12(3), 384–395. https://doi.org/10.1016/S1665-6423(14)71620-7
Selcuk, S. (2017). Predictive maintenance, its implementation and latest trends. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 231(9), 1670–1679. https://doi.org/10.1177/0954405415601640
Shao, S., Member, S., Mcaleer, S., Yan, R., & Member, S. (2018). Highly-Accurate Machine Fault Diagnosis Using. IEEE Transactions on Industrial Informatics, PP(c), 1. https://doi.org/10.1109/TII.2018.2864759
Shen, F., Chen, C., Yan, R., & Gao, R. X. (2016). Bearing fault diagnosis based on SVD feature extraction and transfer learning classification. Proceedings of 2015 Prognostics and System Health Management Conference, PHM 2015, 1. https://doi.org/10.1109/PHM.2015.7380088
Shirkhorshidi, A. S., Aghabozorgi, S., & Ying Wah, T. (2015). A Comparison study on similarity and dissimilarity measures in clustering continuous data. PLoS ONE, 10(12), 1–20. https://doi.org/10.1371/journal.pone.0144059
Sun, C., Ma, M., Zhao, Z., Tian, S., Yan, R., & Chen, X. (2019). Deep Transfer Learning Based on Sparse Autoencoder for Remaining Useful Life Prediction of Tool in Manufacturing. IEEE Transactions on Industrial Informatics, 15(4), 2416–2425. https://doi.org/10.1109/TII.2018.2881543
Szmrecsanyi, B. (2009). Grammatical variation in British English dialects: A study in corpus-based dialectometry. In Grammatical Variation in British English Dialects: A Study in Corpus-Based Dialectometry. https://doi.org/10.1017/CBO9780511763380
Thrun, S. (1997). Multitask Learning *. 75, 41–75.
Thrun, S., & Pratt, L. (1998). Learning to Learn: Introduction and Overview. In Learning to Learn (pp. 3–17). Springer US. https://doi.org/10.1007/978-1-4615-5529-2_1
Tran, V. T., Althobiani, F., & Ball, A. (2014). An approach to fault diagnosis of reciprocating compressor valves using Teager-Kaiser energy operator and deep belief networks. Expert Systems with Applications, 41(9), 4113–4122. https://doi.org/10.1016/j.eswa.2013.12.026
Tsiakmaki, M., Kostopoulos, G., Kotsiantis, S., & Ragos, O. (2020). Transfer learning from deep neural networks for predicting student performance. Applied Sciences (Switzerland), 10(6). https://doi.org/10.3390/app10062145
Tzafestas, S. G., & Dalianis, P. J. (1994). Fault diagnosis in complex systems using artificial neural networks. Proceedings of the IEEE Conference on Control Applications, 2, 877–882. https://doi.org/10.1109/cca.1994.381206
Vakharia, V., Gupta, V. K., & Kankar, P. K. (2015a). Ball Bearing Fault Diagnosis using Supervised and Unsupervised Machine Learning Methods. The International Journal of Acoustics and Vibration, 20(4). https://doi.org/10.20855/ijav.2015.20.4387
Vakharia, V., Gupta, V. K., & Kankar, P. K. (2015b). Ball Bearing Fault Diagnosis using Supervised and Unsupervised Machine Learning Methods. The International Journal of Acoustics and Vibration, 20(4). https://doi.org/10.20855/ijav.2015.20.4387
Xia, M., Li, T., Liu, L., Xu, L., & de Silva, C. W. (2017). Intelligent fault diagnosis approach with unsupervised feature learning by stacked denoising autoencoder. IET Science, Measurement and Technology, 11(6), 687–695. https://doi.org/10.1049/iet-smt.2016.0423
Xia, M., Li, T., Xu, L., Liu, L., & De Silva, C. W. (2018). Fault Diagnosis for Rotating Machinery Using Multiple Sensors and Convolutional Neural Networks. IEEE/ASME Transactions on Mechatronics, 23(1), 101–110. https://doi.org/10.1109/TMECH.2017.2728371
Xu, G., Liu, M., Jiang, Z., Shen, W., & Huang, C. (2019). Online Fault Diagnosis Method Based on Transfer Convolutional Neural Networks. IEEE Transactions on Instrumentation and Measurement, PP, 1–12. https://doi.org/10.1109/tim.2019.2902003
Xu, Y., Sun, Y., Liu, X., & Zheng, Y. (2019). A Digital-Twin-Assisted Fault Diagnosis Using Deep Transfer Learning. IEEE Access, 7, 19990–19999. https://doi.org/10.1109/ACCESS.2018.2890566
Zhang, R. A. N., Tao, H., & Wu, L. (2017). Transfer Learning With Neural Networks for Bearing Fault Diagnosis in Changing Working Conditions. 5, 14347–14357.
Zhang, X., Yu, F. X., Chang, S.-F., & Wang, S. (2015). Deep Transfer Network: Unsupervised Domain Adaptation. http://arxiv.org/abs/1503.00591
Zhang, Y., Hu, X., & Fang, Y. (2010). Logistic regression for transductive transfer learning from multiple sources. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6441 LNAI(PART 2), 175–182. https://doi.org/10.1007/978-3-642-17313-4_18
Zhu, X. (2008). Semi-Supervised Learning Literature Survey Contents.
Section
Technical Papers