Early Fault Detection in Particle Accelerator Power Electronics Using Ensemble Learning
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Jun 11, 2023
Majdi I. Radaideh
Chris Pappas
Mark Wezensky
Pradeep Ramuhalli
Sarah Cousineau
Abstract
Early fault detection and fault prognosis are crucial to ensure efficient and safe operations of complex engineering systems such as the Spallation Neutron Source (SNS) and its power electronics (high voltage converter modulators). Following an advanced experimental facility setup that mimics SNS operating conditions, the authors successfully conducted 21 early fault detection experiments, where fault precursors are introduced in the system to a degree enough to cause degradation in the waveform signals, but not enough to reach a real fault. Nine different machine learning techniques based on ensemble trees, convolutional neural networks, support vector machines, and hierarchical voting ensembles are proposed to detect the fault precursors. Although all 9 models have shown a perfect and identical performance during the training and testing phase, the performance of most models has decreased in the next test phase once they got exposed to realworld data from the 21 experiments. The hierarchical voting ensemble, which features multiple layers of diverse models, maintains a distinguished performance in early detection of the fault precursors with 95% success rate (20/21 tests), followed by adaboost and extremely randomized trees with 52% and 48% success rates, respectively. The support vector machine models were the worst with only 24% success rate (5/21 tests). The study concluded that a successful implementation of machine learning in the SNS or particle accelerator power systems would require a major upgrade in the controller and the data acquisition system to facilitate streaming and handling big data for the machine learning models. In addition, this study shows that the best performing models were diverse and based on the ensemble concept to reduce the bias and hyperparameter sensitivity of individual models.
##plugins.themes.bootstrap3.article.details##
Keywords
Fault Prognosis, Predictive Maintenance, High Voltage Converter Modulator, Decision Trees, Spallation Neutron Source
References
Agasthian, A., Pamula, R., & Kumaraswamidhas, L. A. (2019). Fault classification and detection in wind turbine using cuckoo-optimized support vector machine. Neural Computing and Applications, 31(5), 1503–1511.
Arunthavanathan, R., Khan, F., Ahmed, S., & Imtiaz, S. (2021). A deep learning model for process fault prognosis. Process Safety and Environmental Protection, 154, 467–479.
Belagoune, S., Bali, N., Bakdi, A., Baadji, B., & Atif, K. (2021). Deep learning through lstm classification and regression for transmission line fault detection, diagnosis and location in large-scale multi-machine power systems. Measurement, 177, 109330.
Blokland, W., Ramuhalli, P., Peters, C., Yucesan, Y., Zhukov, A., Schram, M., . . . Jeske, T. (2021). Uncertainty aware anomaly detection to predict errant beam pulses in the sns accelerator. arXiv preprint arXiv:2110.12006.
Bode, G., Thul, S., Baranski, M., & M¨uller, D. (2020). Realworld application of machine-learning-based fault detection trained with experimental data. Energy, 198, 117323.
Breiman, L. (1996). Bagging predictors. Machine learning, 24(2), 123–140.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5–32.
Bukkapatnam, S. T., Afrin, K., Dave, D., & Kumara, S. R. (2019). Machine learning and ai for long-term fault prognosis in complex manufacturing systems. Cirp Annals, 68(1), 459–462.
Cervantes, J., Garcia-Lamont, F., Rodr´ıguez-Mazahua, L., & Lopez, A. (2020). A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing, 408, 189–215.
Edelen, A. L., Biedron, S., Chase, B., Edstrom, D., Milton, S., & Stabile, P. (2016). Neural networks for modeling and control of particle accelerators. IEEE Transactions on Nuclear Science, 63(2), 878–897.
Felsberger, L., Apollonio, A., Cartier-Michaud, T., Müller, A., Todd, B., & Kranzlm¨uller, D. (2020). Explainable deep learning for fault prognostics in complex systems: A particle accelerator use-case. In International crossdomain conference for machine learning and knowledge extraction (pp. 139–158).
Fernandes, M., Corchado, J. M., & Marreiros, G. (2022). Machine learning techniques applied to mechanical fault diagnosis and fault prognosis in the context of real industrial manufacturing use-cases: a systematic literature review. Applied Intelligence, 1–35.
Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1), 119–139.
Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189–1232.
Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine learning, 63(1), 3–42.
Hajji, M., Harkat, M.-F., Kouadri, A., Abodayeh, K., Mansouri, M., Nounou, H., & Nounou, M. (2021). Multivariate feature extraction based supervised machine learning for fault detection and diagnosis in photovoltaic systems. European Journal of Control, 59, 313–321.
Henderson, S., Abraham, W., Aleksandrov, A., Allen, C., Alonso, J., Anderson, D., . . . others (2014). The spallation neutron source accelerator system design. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 763, 610–673.
Kolokas, N., Vafeiadis, T., Ioannidis, D., & Tzovaras, D. (2020). A generic fault prognostics algorithm for manufacturing industries using unsupervised machine learning classifiers. Simulation Modelling Practice and Theory, 103, 102109.
Kozjek, D., Butala, P., et al. (2017). Knowledge elicitation for fault diagnostics in plastic injection moulding: A case for machine-to-machine communication. CIRP annals, 66(1), 433–436.
Liu, C., Tang, D., Zhu, H., & Nie, Q. (2021). A novel predictive maintenance method based on deep adversarial learning in the intelligent manufacturing system. IEEE Access, 9, 49557–49575.
Luo, B., Wang, H., Liu, H., Li, B., & Peng, F. (2018). Early fault detection of machine tools based on deep learning and dynamic identification. IEEE Transactions on Industrial Electronics, 66(1), 509–518.
Mohapatra, D., Subudhi, B., & Daniel, R. (2020). Realtime sensor fault detection in tokamak using different machine learning algorithms. Fusion Engineering and Design, 151, 111401.
Mohd Amiruddin, A. A. A., Zabiri, H., Taqvi, S. A. A., & Tufa, L. D. (2020). Neural network applications in fault diagnosis and detection: an overview of implementations in engineering-related systems. Neural Computing and Applications, 32(2), 447–472.
Nguyen, D., Lee, M., Sass, R., & Shoaee, H. (1991). Accelerator and feedback control simulation using neural networks (Tech. Rep.). Stanford Linear Accelerator Center, Menlo Park, CA (USA).
Noble, W. S. (2006). What is a support vector machine? Nature biotechnology, 24(12), 1565–1567.
Radaideh, M. I., Pappas, C., & Cousineau, S. (2022). Real electronic signal data from particle accelerator power systems for machine learning anomaly detection. Data in Brief , 43, 108473.
Radaideh, M. I., Pappas, C., Walden, J., Lu, D., Vidyaratne, L., Britton, T., . . . Cousineau, S. (2022). Time series anomaly detection in power electronics signals with recurrent and convlstm autoencoders. Digital Signal Processing, 130, 103704.
Rescic, M., Seviour, R., & Blokland, W. (2020). Predicting particle accelerator failures using binary classifiers. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 955, 163240.
Rescic, M., Seviour, R., & Blokland, W. (2022). Improvements of pre-emptive identification of particle accelerator failures using binary classifiers and dimensionality reduction. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 1025, 166064.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostic techniques. In 2008 international conference on prognostics and health management (pp. 1–17).
Scheinker, A. (2021). Adaptive machine learning for robust diagnostics and control of time-varying particle accelerator components and beams. Information, 12(4), 161.
Shao, H., Jiang, H., Wang, F., & Zhao, H. (2017). An enhancement deep feature fusion method for rotating machinery fault diagnosis. Knowledge-Based Systems, 119, 200–220.
Syafrudin, M., Alfian, G., Fitriyani, N. L., & Rhee, J. (2018). Performance analysis of iot-based sensor, big data processing, and machine learning model for real-time monitoring system in automotive manufacturing. Sensors, 18(9), 2946.
Taqvi, S. A., Tufa, L. D., Zabiri, H., Maulud, A. S., & Uddin, F. (2020). Fault detection in distillation column using narx neural network. Neural Computing and Applications, 32(8), 3503–3519.
Vachtsevanos, G. J., & Vachtsevanos, G. J. (2006). Intelligent fault diagnosis and prognosis for engineering systems (Vol. 456). Wiley Online Library.
Wang, L., Zhang, Z., Long, H., Xu, J., & Liu, R. (2016). Wind turbine gearbox failure identification with deep neural networks. IEEE Transactions on Industrial Informatics, 13(3), 1360–1368.
Wang, P., Poovendran, P., & Manokaran, K. B. (2021). Fault detection and control in integrated energy system using machine learning. Sustainable Energy Technologies and Assessments, 47, 101366.
Zhang, L., Lin, J., Liu, B., Zhang, Z., Yan, X., & Wei, M. (2019). A review on deep learning applications in prognostics and health management. Ieee Access, 7, 162415–162438.
Zhang, W., Yang, D., & Wang, H. (2019). Data-driven methods for predictive maintenance of industrial equipment: A survey. IEEE Systems Journal, 13(3), 2213–2227.
Zhang, Y., Beudaert, X., Argandoña, J., Ratchev, S., & Munoa, J. (2020). A cpps based on gbdt for predicting failure events in milling. The International Journal of Advanced Manufacturing Technology, 111(1), 341–357.
Arunthavanathan, R., Khan, F., Ahmed, S., & Imtiaz, S. (2021). A deep learning model for process fault prognosis. Process Safety and Environmental Protection, 154, 467–479.
Belagoune, S., Bali, N., Bakdi, A., Baadji, B., & Atif, K. (2021). Deep learning through lstm classification and regression for transmission line fault detection, diagnosis and location in large-scale multi-machine power systems. Measurement, 177, 109330.
Blokland, W., Ramuhalli, P., Peters, C., Yucesan, Y., Zhukov, A., Schram, M., . . . Jeske, T. (2021). Uncertainty aware anomaly detection to predict errant beam pulses in the sns accelerator. arXiv preprint arXiv:2110.12006.
Bode, G., Thul, S., Baranski, M., & M¨uller, D. (2020). Realworld application of machine-learning-based fault detection trained with experimental data. Energy, 198, 117323.
Breiman, L. (1996). Bagging predictors. Machine learning, 24(2), 123–140.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5–32.
Bukkapatnam, S. T., Afrin, K., Dave, D., & Kumara, S. R. (2019). Machine learning and ai for long-term fault prognosis in complex manufacturing systems. Cirp Annals, 68(1), 459–462.
Cervantes, J., Garcia-Lamont, F., Rodr´ıguez-Mazahua, L., & Lopez, A. (2020). A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing, 408, 189–215.
Edelen, A. L., Biedron, S., Chase, B., Edstrom, D., Milton, S., & Stabile, P. (2016). Neural networks for modeling and control of particle accelerators. IEEE Transactions on Nuclear Science, 63(2), 878–897.
Felsberger, L., Apollonio, A., Cartier-Michaud, T., Müller, A., Todd, B., & Kranzlm¨uller, D. (2020). Explainable deep learning for fault prognostics in complex systems: A particle accelerator use-case. In International crossdomain conference for machine learning and knowledge extraction (pp. 139–158).
Fernandes, M., Corchado, J. M., & Marreiros, G. (2022). Machine learning techniques applied to mechanical fault diagnosis and fault prognosis in the context of real industrial manufacturing use-cases: a systematic literature review. Applied Intelligence, 1–35.
Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1), 119–139.
Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189–1232.
Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine learning, 63(1), 3–42.
Hajji, M., Harkat, M.-F., Kouadri, A., Abodayeh, K., Mansouri, M., Nounou, H., & Nounou, M. (2021). Multivariate feature extraction based supervised machine learning for fault detection and diagnosis in photovoltaic systems. European Journal of Control, 59, 313–321.
Henderson, S., Abraham, W., Aleksandrov, A., Allen, C., Alonso, J., Anderson, D., . . . others (2014). The spallation neutron source accelerator system design. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 763, 610–673.
Kolokas, N., Vafeiadis, T., Ioannidis, D., & Tzovaras, D. (2020). A generic fault prognostics algorithm for manufacturing industries using unsupervised machine learning classifiers. Simulation Modelling Practice and Theory, 103, 102109.
Kozjek, D., Butala, P., et al. (2017). Knowledge elicitation for fault diagnostics in plastic injection moulding: A case for machine-to-machine communication. CIRP annals, 66(1), 433–436.
Liu, C., Tang, D., Zhu, H., & Nie, Q. (2021). A novel predictive maintenance method based on deep adversarial learning in the intelligent manufacturing system. IEEE Access, 9, 49557–49575.
Luo, B., Wang, H., Liu, H., Li, B., & Peng, F. (2018). Early fault detection of machine tools based on deep learning and dynamic identification. IEEE Transactions on Industrial Electronics, 66(1), 509–518.
Mohapatra, D., Subudhi, B., & Daniel, R. (2020). Realtime sensor fault detection in tokamak using different machine learning algorithms. Fusion Engineering and Design, 151, 111401.
Mohd Amiruddin, A. A. A., Zabiri, H., Taqvi, S. A. A., & Tufa, L. D. (2020). Neural network applications in fault diagnosis and detection: an overview of implementations in engineering-related systems. Neural Computing and Applications, 32(2), 447–472.
Nguyen, D., Lee, M., Sass, R., & Shoaee, H. (1991). Accelerator and feedback control simulation using neural networks (Tech. Rep.). Stanford Linear Accelerator Center, Menlo Park, CA (USA).
Noble, W. S. (2006). What is a support vector machine? Nature biotechnology, 24(12), 1565–1567.
Radaideh, M. I., Pappas, C., & Cousineau, S. (2022). Real electronic signal data from particle accelerator power systems for machine learning anomaly detection. Data in Brief , 43, 108473.
Radaideh, M. I., Pappas, C., Walden, J., Lu, D., Vidyaratne, L., Britton, T., . . . Cousineau, S. (2022). Time series anomaly detection in power electronics signals with recurrent and convlstm autoencoders. Digital Signal Processing, 130, 103704.
Rescic, M., Seviour, R., & Blokland, W. (2020). Predicting particle accelerator failures using binary classifiers. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 955, 163240.
Rescic, M., Seviour, R., & Blokland, W. (2022). Improvements of pre-emptive identification of particle accelerator failures using binary classifiers and dimensionality reduction. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 1025, 166064.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for evaluating performance of prognostic techniques. In 2008 international conference on prognostics and health management (pp. 1–17).
Scheinker, A. (2021). Adaptive machine learning for robust diagnostics and control of time-varying particle accelerator components and beams. Information, 12(4), 161.
Shao, H., Jiang, H., Wang, F., & Zhao, H. (2017). An enhancement deep feature fusion method for rotating machinery fault diagnosis. Knowledge-Based Systems, 119, 200–220.
Syafrudin, M., Alfian, G., Fitriyani, N. L., & Rhee, J. (2018). Performance analysis of iot-based sensor, big data processing, and machine learning model for real-time monitoring system in automotive manufacturing. Sensors, 18(9), 2946.
Taqvi, S. A., Tufa, L. D., Zabiri, H., Maulud, A. S., & Uddin, F. (2020). Fault detection in distillation column using narx neural network. Neural Computing and Applications, 32(8), 3503–3519.
Vachtsevanos, G. J., & Vachtsevanos, G. J. (2006). Intelligent fault diagnosis and prognosis for engineering systems (Vol. 456). Wiley Online Library.
Wang, L., Zhang, Z., Long, H., Xu, J., & Liu, R. (2016). Wind turbine gearbox failure identification with deep neural networks. IEEE Transactions on Industrial Informatics, 13(3), 1360–1368.
Wang, P., Poovendran, P., & Manokaran, K. B. (2021). Fault detection and control in integrated energy system using machine learning. Sustainable Energy Technologies and Assessments, 47, 101366.
Zhang, L., Lin, J., Liu, B., Zhang, Z., Yan, X., & Wei, M. (2019). A review on deep learning applications in prognostics and health management. Ieee Access, 7, 162415–162438.
Zhang, W., Yang, D., & Wang, H. (2019). Data-driven methods for predictive maintenance of industrial equipment: A survey. IEEE Systems Journal, 13(3), 2213–2227.
Zhang, Y., Beudaert, X., Argandoña, J., Ratchev, S., & Munoa, J. (2020). A cpps based on gbdt for predicting failure events in milling. The International Journal of Advanced Manufacturing Technology, 111(1), 341–357.
Section
Technical Papers