Domain Adaptation for One-Class Classification: Monitoring the Health of Critical Systems Under Limited Information
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Dec 2, 2019
Gabriel Michau
Olga Fink
Abstract
The failure of a complex and safety critical industrial asset can have extremely high consequences. Close monitoring for early detection of abnormal system conditions is therefore required. Data-driven solutions to this problem have been limited for two reasons: First, safety critical assets are
designed and maintained to be highly reliable and faults are rare. Fault detection can thus not be solved with supervised learning. Second, complex industrial systems usually have long lifetime during which they face very different operating conditions. In the early life of the system, the collected data is probably not representative of future operating conditions, making it challenging to train a robust model. In this paper, we propose a methodology to monitor the systems in their early life. To do so, we enhance the training dataset with other units from a fleet, for which longer observations are available. Since each unit has its own specificity, we propose to extract features made independent of their origin by three unsupervised feature alignment techniques. First, using a variational encoder, we impose a shared probabilistic encoder/decoder for both units. Second, we introduce a new loss designed to conserve inter-point spacial relationships between the input and the learned features. Last, we
propose to train in an adversarial manner a discriminator on the origin of the features. Once aligned, the features are fed to a one-class classifier to monitor the health of the system. By exploring the different combinations of the proposed alignment strategies, and by testing them on a real case study, a fleet composed of 112 power plants operated in different geographical locations and under very different operating regimes, we demonstrate that this alignment is necessary and beneficial.
##plugins.themes.bootstrap3.article.details##
Keywords
unsupervised fault detection, domain alignment, feature alignment, fleet, data-driven
References
Arjovsky, M., Chintala, S. & Bottou, L. (2017). Wasserstein generative adversarial networks. In International conference on machine learning (pp. 214–223).
Borgwardt, K. M., Gretton, A., Rasch, M. J., Kriegel, H.-P., Sch¨olkopf, B. & Smola, A. J. (2006, July 15). Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics, 22(14), e49–e57. doi:10.1093/bioinformatics/btl242
Al-Dahidi, S., Di Maio, F., Baraldi, P., Zio, E. & Seraoui, R. (2018). A framework for reconciliating data clusters from a fleet of nuclear power plants turbines for fault diagnosis. Applied Soft Computing, 69, 213–231.
Domingos, P. (2012, October). A Few Useful Things to Know About Machine Learning. Communications of the ACM, Vol. 55(10), 78–87. Retrieved February 14, 2019, from https://cacm.acm.org/magazines/2012/10/155531-afew-useful-things-to-know-about-machine-learning/fulltext
Ellefsen, A. L., Bjørlykhaug, E., Æsøy, V., Ushakov, S. & Zhang, H. (2019). Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture. Reliability Engineering&System Safety, 183, 240–251.
Fan, J., Wang, W. & Zhang, H. (2017). AutoEncoder based high-dimensional data fault detection system. In 2017 ieee 15th international conference on industrial informatics (indin) (pp. 1001–1006). IEEE. doi:10.1109/INDIN. 2017.8104910
Fernando, B., Habrard, A., Sebban, M.&Tuytelaars, T. (2013). Unsupervised visual domain adaptation using subspace alignment. In Proceedings of the ieee international conference on computer vision (pp. 2960–2967).
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., . . . Lempitsky, V. (2016). Domainadversarial training of neural networks. The Journal of Machine Learning Research, 17(1), 2096–2030.
González-Pri Da, V., Orchard, M., Martí, C., Guillén, A., Shambhu, J. & Shariff, S. (2016). Case Study based on Inequality Indices for the Assessments of Industrial Fleets, 250–255. doi:10.1016/j.ifacol.2016.11.043
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. (2017). Improved training of wasserstein gans. In Advances in neural information processing systems (pp. 5767–5777).
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., . . . Lerchner, A. (2017). Beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations (Vol. 3).
Huang, G.-B., Chen, L., Siew, C. K. et al. (2006). Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Networks, 17(4), 879–892. Retrieved March 7, 2017, from http://ieeexplore.ieee.org/iel5/72/34606/01650244.pdf
Jin, C., Djurdjanovic, D., Ardakani, H. D., Wang, K., Buzza, M., Begheri, B., . . . Lee, J. (2015). A comprehensive framework of factory-to-factory dynamic fleet-level prognostics and operation management for geographically distributed assets. In 2015 ieee international conference on automation science and engineering (case) (pp. 225–230). IEEE. doi:10.1109/CoASE.2015.7294066
Kim, D., Yang, H., Chung, M., Cho, S., Kim, H., Kim, M., . . .Kim, E. (2018). Squeezed convolutional variational autoencoder for unsupervised anomaly detection in edge device industrial internet of things. In 2018 international conference on information and computer technologies (icict) (pp. 67–71). IEEE.
Kingma, D. P.&Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Kingma, D. P. & Welling, M. (2013). Auto-encoding variational bayes. arXiv: 1312.6114.
Kouw, W. M. & Loog, M. (2019). A review of single-source unsupervised domain adaptation. arXiv: 1901.05335.
Lapira, E. R. (2012). Fault detection in a network of similar machines using clustering approach (Doctoral dissertation, University of Cincinnati).
Leng, Q., Qi, H., Miao, J., Zhu, W. & Su, G. (2015). Oneclass classification with extreme learning machine. Mathematical problems in engineering, 2015.
Leone, G. [G.], Cristaldi, L. & Turrin, S. (2016). A datadriven prognostic approach based on sub-fleet knowledge extraction. In 14th imeko tc10 workshop on technical diagnostics 2016: New perspectives in measurements, tools and techniques for systems reliability, maintainability and safety.
Leone, G. [Giacomo], Cristaldi, L. & Turrin, S. (2017). A data-driven prognostic approach based on statistical similarity: An application to industrial circuit breakers. Measurement, 108, 163–170. doi:10.1016/J.MEASUREMENT. 2017.02.017
Li, X., Zhang, W. & Ding, Q. (2018). Cross-Domain Fault Diagnosis of Rolling Element Bearings Using Deep Generative Neural Networks. IEEE Transactions on Industrial Electronics, 1–1. doi:10.1109/TIE.2018.2868023
Liu, Z. (2018). Cyber-physical system augmented prognostics and health management for fleet-based systems (Doctoral dissertation, University of Cincinnati).
Lu, W., Liang, B., Cheng, Y., Meng, D., Yang, J. & Zhang, T. (2017). Deep model based domain adaptation for fault diagnosis. IEEE Transactions on Industrial Electronics, 64(3), 2296–2305.
Margolis, A. (2011). A literature review of domain adaptation with unlabeled data. Tec. Report, 1–42.
Michau, G. & Fink, O. (2019, June 17–19). Unsupervised Fault Detection in Varying Operating Conditions. In 2019 IEEE International Conference on Prognostics and Health Management. 2019 IEEE International Conference on Prognostics and Health Management. San Francisco, CA, USA.
Michau, G., Hu, Y., Palmé, T. & Fink, O. (2019, August 24). Feature learning for fault detection in high-dimensional condition monitoring signals. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability. doi:10.1177/1748006X19868335
Michau, G., Palmé, T. & Fink, O. (2017, October). Deep Feature Learning Network for Fault Detection and Isolation. In Annual Conference of the PHM Society 2017. Annual Conference of the Prognostics and Health Management Society 2017. St. Petersburg, Florida.
Michau, G., Palmé, T. & Fink, O. (2018). Fleet PHM for Critical Systems: Bi-level Deep Learning Approach for Fault Detection. In Proceedings of the European Conference of the PHM Society (Vol. 4). Fourth european conference of the prognostics and health management society 2018. Utrecht, The Netherlands.
Pecht, M. G. & Kang, M. (2019). Machine Learning: Anomaly Detection. In Prognostics and Health Management of Electronics: Fundamentals, Machine Learning, and the Internet of Things. IEEE. doi:10 . 1002 / 9781119515326.ch6
Peysson, F., Mozzati, C., Leon, D., Lafuste, Q. & Leger, J.-B. (2019). Fleet-Wide Proactive Maintenance of Machine Tools. In Twin-control (pp. 209–224). Cham: Springer International Publishing. doi:10 . 1007 / 978 - 3 - 030 - 02203-7 13
Wang, Q., Michau, G. & Fink, O. (2019, May 5). Domain adaptive transfer learning for fault diagnosis. 2019 Prognostics and System Health Management Conference, arXiv:1905.06004.
Wu, Y., Winston, E., Kaushik, D. & Lipton, Z. (2019). Domain adaptation with asymmetrically-relaxed distribution alignment. arXiv: 1903.01689.
Xie, J., Zhang, L., Duan, L. & Wang, J. (2016). On crossdomain feature fusion in gearbox fault diagnosis under various operating conditions based on transfer component analysis. In Prognostics and Health Management (ICPHM), 2016 IEEE International Conference on (pp. 1–6). IEEE.
Yan,W. (2016). One-class extreme learning machines for gas turbine combustor anomaly detection. In 2016 international joint conference on neural networks (ijcnn) (pp. 2909–2914). IEEE.
Yoon, A. S., Lee, T., Lim, Y., Jung, D., Kang, P., Kim, D., . . . Choi, Y. (2017). Semi-supervised learning with deep generative models for asset failure prediction. arXiv: 1709.00845.
Zhang, R., Tao, H.,Wu, L.&Guan, Y. (2017). Transfer Learning With Neural Networks for Bearing Fault Diagnosis in ChangingWorking Conditions. IEEE Access, 5, 14347–14357. doi:10.1109/ACCESS.2017.2720965
Zio, E. & Di Maio, F. (2010). A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system. Reliability Engineering & System Safety, 95(1), 49–57.
Borgwardt, K. M., Gretton, A., Rasch, M. J., Kriegel, H.-P., Sch¨olkopf, B. & Smola, A. J. (2006, July 15). Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics, 22(14), e49–e57. doi:10.1093/bioinformatics/btl242
Al-Dahidi, S., Di Maio, F., Baraldi, P., Zio, E. & Seraoui, R. (2018). A framework for reconciliating data clusters from a fleet of nuclear power plants turbines for fault diagnosis. Applied Soft Computing, 69, 213–231.
Domingos, P. (2012, October). A Few Useful Things to Know About Machine Learning. Communications of the ACM, Vol. 55(10), 78–87. Retrieved February 14, 2019, from https://cacm.acm.org/magazines/2012/10/155531-afew-useful-things-to-know-about-machine-learning/fulltext
Ellefsen, A. L., Bjørlykhaug, E., Æsøy, V., Ushakov, S. & Zhang, H. (2019). Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture. Reliability Engineering&System Safety, 183, 240–251.
Fan, J., Wang, W. & Zhang, H. (2017). AutoEncoder based high-dimensional data fault detection system. In 2017 ieee 15th international conference on industrial informatics (indin) (pp. 1001–1006). IEEE. doi:10.1109/INDIN. 2017.8104910
Fernando, B., Habrard, A., Sebban, M.&Tuytelaars, T. (2013). Unsupervised visual domain adaptation using subspace alignment. In Proceedings of the ieee international conference on computer vision (pp. 2960–2967).
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., . . . Lempitsky, V. (2016). Domainadversarial training of neural networks. The Journal of Machine Learning Research, 17(1), 2096–2030.
González-Pri Da, V., Orchard, M., Martí, C., Guillén, A., Shambhu, J. & Shariff, S. (2016). Case Study based on Inequality Indices for the Assessments of Industrial Fleets, 250–255. doi:10.1016/j.ifacol.2016.11.043
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C. (2017). Improved training of wasserstein gans. In Advances in neural information processing systems (pp. 5767–5777).
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., . . . Lerchner, A. (2017). Beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations (Vol. 3).
Huang, G.-B., Chen, L., Siew, C. K. et al. (2006). Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Networks, 17(4), 879–892. Retrieved March 7, 2017, from http://ieeexplore.ieee.org/iel5/72/34606/01650244.pdf
Jin, C., Djurdjanovic, D., Ardakani, H. D., Wang, K., Buzza, M., Begheri, B., . . . Lee, J. (2015). A comprehensive framework of factory-to-factory dynamic fleet-level prognostics and operation management for geographically distributed assets. In 2015 ieee international conference on automation science and engineering (case) (pp. 225–230). IEEE. doi:10.1109/CoASE.2015.7294066
Kim, D., Yang, H., Chung, M., Cho, S., Kim, H., Kim, M., . . .Kim, E. (2018). Squeezed convolutional variational autoencoder for unsupervised anomaly detection in edge device industrial internet of things. In 2018 international conference on information and computer technologies (icict) (pp. 67–71). IEEE.
Kingma, D. P.&Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Kingma, D. P. & Welling, M. (2013). Auto-encoding variational bayes. arXiv: 1312.6114.
Kouw, W. M. & Loog, M. (2019). A review of single-source unsupervised domain adaptation. arXiv: 1901.05335.
Lapira, E. R. (2012). Fault detection in a network of similar machines using clustering approach (Doctoral dissertation, University of Cincinnati).
Leng, Q., Qi, H., Miao, J., Zhu, W. & Su, G. (2015). Oneclass classification with extreme learning machine. Mathematical problems in engineering, 2015.
Leone, G. [G.], Cristaldi, L. & Turrin, S. (2016). A datadriven prognostic approach based on sub-fleet knowledge extraction. In 14th imeko tc10 workshop on technical diagnostics 2016: New perspectives in measurements, tools and techniques for systems reliability, maintainability and safety.
Leone, G. [Giacomo], Cristaldi, L. & Turrin, S. (2017). A data-driven prognostic approach based on statistical similarity: An application to industrial circuit breakers. Measurement, 108, 163–170. doi:10.1016/J.MEASUREMENT. 2017.02.017
Li, X., Zhang, W. & Ding, Q. (2018). Cross-Domain Fault Diagnosis of Rolling Element Bearings Using Deep Generative Neural Networks. IEEE Transactions on Industrial Electronics, 1–1. doi:10.1109/TIE.2018.2868023
Liu, Z. (2018). Cyber-physical system augmented prognostics and health management for fleet-based systems (Doctoral dissertation, University of Cincinnati).
Lu, W., Liang, B., Cheng, Y., Meng, D., Yang, J. & Zhang, T. (2017). Deep model based domain adaptation for fault diagnosis. IEEE Transactions on Industrial Electronics, 64(3), 2296–2305.
Margolis, A. (2011). A literature review of domain adaptation with unlabeled data. Tec. Report, 1–42.
Michau, G. & Fink, O. (2019, June 17–19). Unsupervised Fault Detection in Varying Operating Conditions. In 2019 IEEE International Conference on Prognostics and Health Management. 2019 IEEE International Conference on Prognostics and Health Management. San Francisco, CA, USA.
Michau, G., Hu, Y., Palmé, T. & Fink, O. (2019, August 24). Feature learning for fault detection in high-dimensional condition monitoring signals. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability. doi:10.1177/1748006X19868335
Michau, G., Palmé, T. & Fink, O. (2017, October). Deep Feature Learning Network for Fault Detection and Isolation. In Annual Conference of the PHM Society 2017. Annual Conference of the Prognostics and Health Management Society 2017. St. Petersburg, Florida.
Michau, G., Palmé, T. & Fink, O. (2018). Fleet PHM for Critical Systems: Bi-level Deep Learning Approach for Fault Detection. In Proceedings of the European Conference of the PHM Society (Vol. 4). Fourth european conference of the prognostics and health management society 2018. Utrecht, The Netherlands.
Pecht, M. G. & Kang, M. (2019). Machine Learning: Anomaly Detection. In Prognostics and Health Management of Electronics: Fundamentals, Machine Learning, and the Internet of Things. IEEE. doi:10 . 1002 / 9781119515326.ch6
Peysson, F., Mozzati, C., Leon, D., Lafuste, Q. & Leger, J.-B. (2019). Fleet-Wide Proactive Maintenance of Machine Tools. In Twin-control (pp. 209–224). Cham: Springer International Publishing. doi:10 . 1007 / 978 - 3 - 030 - 02203-7 13
Wang, Q., Michau, G. & Fink, O. (2019, May 5). Domain adaptive transfer learning for fault diagnosis. 2019 Prognostics and System Health Management Conference, arXiv:1905.06004.
Wu, Y., Winston, E., Kaushik, D. & Lipton, Z. (2019). Domain adaptation with asymmetrically-relaxed distribution alignment. arXiv: 1903.01689.
Xie, J., Zhang, L., Duan, L. & Wang, J. (2016). On crossdomain feature fusion in gearbox fault diagnosis under various operating conditions based on transfer component analysis. In Prognostics and Health Management (ICPHM), 2016 IEEE International Conference on (pp. 1–6). IEEE.
Yan,W. (2016). One-class extreme learning machines for gas turbine combustor anomaly detection. In 2016 international joint conference on neural networks (ijcnn) (pp. 2909–2914). IEEE.
Yoon, A. S., Lee, T., Lim, Y., Jung, D., Kang, P., Kim, D., . . . Choi, Y. (2017). Semi-supervised learning with deep generative models for asset failure prediction. arXiv: 1709.00845.
Zhang, R., Tao, H.,Wu, L.&Guan, Y. (2017). Transfer Learning With Neural Networks for Bearing Fault Diagnosis in ChangingWorking Conditions. IEEE Access, 5, 14347–14357. doi:10.1109/ACCESS.2017.2720965
Zio, E. & Di Maio, F. (2010). A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system. Reliability Engineering & System Safety, 95(1), 49–57.
Issue
Section
Technical Papers