Multi-label Prediction in Time Series Data using Deep Neural Networks
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Wenyu Zhang
Devesh K. Jha
Emil Laftchiev
Daniel Nikovski
Abstract
This paper addresses a multi-label predictive fault classification problem for multidimensional time-series data. While fault (event) detection problems have been thoroughly studied in literature, most of the state-of-the-art techniques can’t reliably predict faults (events) over a desired future horizon. In the most general setting of these types of problems, one or more samples of data across multiple time series can be assigned several concurrent fault labels from a finite, known set and the task is to predict the possibility of fault occurrence over a desired time horizon. This type of problem is usually accompanied by strong class imbalances where some classes are represented by only a few samples. Importantly, in many applications of the problem such as fault prediction and predictive maintenance, it is exactly these rare classes that are of most interest. To address the problem, this paper proposes a general approach that utilizes a multi-label recurrent neural network with a new cost function that accentuates learning in the imbalanced classes. The proposed algorithm is tested on two public benchmark datasets: an industrial plant dataset
from the PHM Society Data Challenge, and a human activity recognition dataset. The results are compared with state-ofthe-art techniques for time-series classification and evaluation is performed using the F1-score, precision and recall.
##plugins.themes.bootstrap3.article.details##
Keywords
Condition Monitoring, Prognostics, Deep Learning, Time Series Embeddings, Recurrent Neural Networks, Anomaly Prediction, Anomaly Localization, Anomaly Diagnosis
References
2015 PHM Data Challenge. (2015). https://www.phmsociety.org/events/conference/phm/15/data-challenge. (Accessed: 2018-09-20)
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 15.
Chavarriaga, R., Sagha, H., Roggen, D., & Ferscha, A. (2011). Opportunity Activity Recognition Challenge: Results and Conclusions.
http://www.opportunity-project.eu/system/files/docs/Workshop/2011SMCChallenge.pdf. (Accessed: 2019-08-27)
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010, Oct 01). Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing, 14(7), 645–662.
Heimes, F. O. (2008, Oct). Recurrent neural networks for remaining useful life estimation. In 2008 International Conference on Prognostics and Health Management (p. 1-6).
Holst, A., Bohlin, M., Ekman, J., Sellin, O., Lindstr¨om, B., & Larsen, S. (2012). Statistical anomaly detection for train fleets. AI Magazine, 34, 33-42.
Hu, C., Zhou, Z., Zhang, J., & Si, X. (2015). A survey on life prediction of equipment. Chinese Journal of Aeronautics, 28(1), 25 - 33.
Jain, H., Prabhu, Y., & Varma, M. (2016). Extreme multilabel loss functions for recommendation, tagging, ranking, other missing label applications. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 935–944). New York, NY, USA: ACM.
Jha, D. K., Srivastav, A., & Ray, A. (2016). Temporal learning in video data using deep learning and gaussian processes. In Workshop on Machine Learning for Prognostics and Health Managament at 2016 KDD, San Francisco, CA.
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering System Safety, 172, 1 - 11.
Lipton, Z. C., Kale, D. C., Elkan, C., &Wetzel, R. C. (2016). Learning to diagnose with LSTM recurrent neural networks. International Conference on Learning Representations.
Liu, Q., Wu, S., Wang, L., & Tan, T. (2016). Predicting the next location: A recurrent model with spatial and temporal contexts. In AAAI Conference on Artificial Intelligence.
Liu, Z., Zhang, W., Lin, S., & Quek, T. Q. S. (2017, April). Heterogeneous sensor data fusion by deep multimodal encoding. IEEE Journal of Selected Topics in Signal Processing, 11(3), 479-491.
McClure, N. (2017). Tensorflow machine learning cookbook. Packt Publishing.
Molaei, S. M., & Keyvanpour, M. R. (2015, March). An analytical review for event prediction system on time series. In 2015 2nd International Conference on Pattern Recognition and Image Analysis (IPRIA) (p. 1-6).
Mueller, J., & Thyagarajan, A. (2016). Siamese recurrent architectures for learning sentence similarity. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (pp. 2786–2792). AAAI Press.
Neculoiu, P., Versteegh, M., & Rotaru, M. (2016). Learning text similarity with Siamese recurrent networks. In Rep4NLP@ACL.
Ordóñez, F. J., & Roggen, D. (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1).
Patri, O. P., Sharma, A. B., Chen, H., Jiang, G., Panangadan, A. V., & Prasanna, V. K. (2014, Oct). Extracting discriminative shapelets from heterogeneous sensor data. In 2014 IEEE International Conference on Big Data (p. 1095-1104).
Pei, W., Tax, D. M. J., & van der Maaten, L. (2016). Modeling time series similarity with Siamese recurrent networks. CoRR, abs/1603.04713.
Read, J., Martino, L., & Hollm´en, J. (2017). Multi-label methods for prediction with sequential data. Pattern Recognition, 63, 45 - 55.
Read, J., Pfahringer, B., Holmes, G., & Frank, E. (2009). Classifier chains for multi-label classification. In Proceedings of the 2009th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II (pp. 254–269). Berlin, Heidelberg: Springer-Verlag.
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Frster, K., Trster, G., . . . d. R. Milln, J. (2010, June). Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh International Conference on Networked Sensing Systems (INSS) (p. 233-240).
Romeres, D., Jha, D. K., Yerazunis,W., Nikovski, D., & Dau, H. A. (2019, June). Anomaly detection for insertion tasks in robotic assembly using Gaussian process models. In 2019 18th European Control Conference (ECC) (p. 1017-1022). doi: 10.23919/ECC.2019.8795698
Ryoo, M. S. (2011, Nov). Human activity prediction: Early recognition of ongoing activities from streaming videos. In 2011 International Conference on Computer Vision (p. 1036-1043).
Salfner, F., Lenk, M., & Malek, M. (2010, March). A survey of online failure prediction methods. ACM Comput. Surv., 42(3), 10:1–10:42.
Sarkar, S., Jha, D. K., Lore, K. G., Sarkar, S., & Ray, A. (2016). Multimodal spatiotemporal information fusion using neural-symbolic modeling for early detection of combustion instabilities. In 2016 American Control Conference (ACC) (pp. 4918–4923).
Seto, S., Zhang, W., & Zhou, Y. (2015, Dec). Multivariate time series classification using dynamic time warping template selection for human activity recognition. In 2015 IEEE Symposium Series on Computational Intelligence (p. 1399-1406).
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (pp. 3104–3112). Cambridge, MA, USA: MIT Press.
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2011, July). Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering, 23(7), 1079-1089. Understanding LSTMs. (2015). http://colah.github.io/posts/2015-08-Understanding-LSTMs/. (Accessed: 2018-09-20)
Welte, T. M., & Wang, K. (2014, Mar 01). Models for lifetime estimation: An overview with focus on applications to wind turbines. Advances in Manufacturing, 2(1), 79–87.
Xiao, W. (2016, 03). A probabilistic machine learning approach to detect industrial plant faults. International Journal of Prognostics and Health Management, 7, 11.
Xie, C., Yang, D., Huang, Y., & Sun, D. (2016). Feature extraction and ensemble decision tree classifier in plant failure detection. International Journal of Prognostics and Health Management, 7.
Yamanishi, K., & Takeuchi, J.-i. (2002). A unifying framework for detecting outliers and change points from non-stationary time series data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 676–681).
Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017, June). Long Short-Term Memory network for remaining useful life estimation. In 2017 IEEE International Conference on Prognostics and Health Management (ICPHM) (p. 88-95).
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 15.
Chavarriaga, R., Sagha, H., Roggen, D., & Ferscha, A. (2011). Opportunity Activity Recognition Challenge: Results and Conclusions.
http://www.opportunity-project.eu/system/files/docs/Workshop/2011SMCChallenge.pdf. (Accessed: 2019-08-27)
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010, Oct 01). Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing, 14(7), 645–662.
Heimes, F. O. (2008, Oct). Recurrent neural networks for remaining useful life estimation. In 2008 International Conference on Prognostics and Health Management (p. 1-6).
Holst, A., Bohlin, M., Ekman, J., Sellin, O., Lindstr¨om, B., & Larsen, S. (2012). Statistical anomaly detection for train fleets. AI Magazine, 34, 33-42.
Hu, C., Zhou, Z., Zhang, J., & Si, X. (2015). A survey on life prediction of equipment. Chinese Journal of Aeronautics, 28(1), 25 - 33.
Jain, H., Prabhu, Y., & Varma, M. (2016). Extreme multilabel loss functions for recommendation, tagging, ranking, other missing label applications. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 935–944). New York, NY, USA: ACM.
Jha, D. K., Srivastav, A., & Ray, A. (2016). Temporal learning in video data using deep learning and gaussian processes. In Workshop on Machine Learning for Prognostics and Health Managament at 2016 KDD, San Francisco, CA.
Li, X., Ding, Q., & Sun, J.-Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering System Safety, 172, 1 - 11.
Lipton, Z. C., Kale, D. C., Elkan, C., &Wetzel, R. C. (2016). Learning to diagnose with LSTM recurrent neural networks. International Conference on Learning Representations.
Liu, Q., Wu, S., Wang, L., & Tan, T. (2016). Predicting the next location: A recurrent model with spatial and temporal contexts. In AAAI Conference on Artificial Intelligence.
Liu, Z., Zhang, W., Lin, S., & Quek, T. Q. S. (2017, April). Heterogeneous sensor data fusion by deep multimodal encoding. IEEE Journal of Selected Topics in Signal Processing, 11(3), 479-491.
McClure, N. (2017). Tensorflow machine learning cookbook. Packt Publishing.
Molaei, S. M., & Keyvanpour, M. R. (2015, March). An analytical review for event prediction system on time series. In 2015 2nd International Conference on Pattern Recognition and Image Analysis (IPRIA) (p. 1-6).
Mueller, J., & Thyagarajan, A. (2016). Siamese recurrent architectures for learning sentence similarity. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (pp. 2786–2792). AAAI Press.
Neculoiu, P., Versteegh, M., & Rotaru, M. (2016). Learning text similarity with Siamese recurrent networks. In Rep4NLP@ACL.
Ordóñez, F. J., & Roggen, D. (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1).
Patri, O. P., Sharma, A. B., Chen, H., Jiang, G., Panangadan, A. V., & Prasanna, V. K. (2014, Oct). Extracting discriminative shapelets from heterogeneous sensor data. In 2014 IEEE International Conference on Big Data (p. 1095-1104).
Pei, W., Tax, D. M. J., & van der Maaten, L. (2016). Modeling time series similarity with Siamese recurrent networks. CoRR, abs/1603.04713.
Read, J., Martino, L., & Hollm´en, J. (2017). Multi-label methods for prediction with sequential data. Pattern Recognition, 63, 45 - 55.
Read, J., Pfahringer, B., Holmes, G., & Frank, E. (2009). Classifier chains for multi-label classification. In Proceedings of the 2009th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II (pp. 254–269). Berlin, Heidelberg: Springer-Verlag.
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Frster, K., Trster, G., . . . d. R. Milln, J. (2010, June). Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh International Conference on Networked Sensing Systems (INSS) (p. 233-240).
Romeres, D., Jha, D. K., Yerazunis,W., Nikovski, D., & Dau, H. A. (2019, June). Anomaly detection for insertion tasks in robotic assembly using Gaussian process models. In 2019 18th European Control Conference (ECC) (p. 1017-1022). doi: 10.23919/ECC.2019.8795698
Ryoo, M. S. (2011, Nov). Human activity prediction: Early recognition of ongoing activities from streaming videos. In 2011 International Conference on Computer Vision (p. 1036-1043).
Salfner, F., Lenk, M., & Malek, M. (2010, March). A survey of online failure prediction methods. ACM Comput. Surv., 42(3), 10:1–10:42.
Sarkar, S., Jha, D. K., Lore, K. G., Sarkar, S., & Ray, A. (2016). Multimodal spatiotemporal information fusion using neural-symbolic modeling for early detection of combustion instabilities. In 2016 American Control Conference (ACC) (pp. 4918–4923).
Seto, S., Zhang, W., & Zhou, Y. (2015, Dec). Multivariate time series classification using dynamic time warping template selection for human activity recognition. In 2015 IEEE Symposium Series on Computational Intelligence (p. 1399-1406).
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (pp. 3104–3112). Cambridge, MA, USA: MIT Press.
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2011, July). Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering, 23(7), 1079-1089. Understanding LSTMs. (2015). http://colah.github.io/posts/2015-08-Understanding-LSTMs/. (Accessed: 2018-09-20)
Welte, T. M., & Wang, K. (2014, Mar 01). Models for lifetime estimation: An overview with focus on applications to wind turbines. Advances in Manufacturing, 2(1), 79–87.
Xiao, W. (2016, 03). A probabilistic machine learning approach to detect industrial plant faults. International Journal of Prognostics and Health Management, 7, 11.
Xie, C., Yang, D., Huang, Y., & Sun, D. (2016). Feature extraction and ensemble decision tree classifier in plant failure detection. International Journal of Prognostics and Health Management, 7.
Yamanishi, K., & Takeuchi, J.-i. (2002). A unifying framework for detecting outliers and change points from non-stationary time series data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 676–681).
Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017, June). Long Short-Term Memory network for remaining useful life estimation. In 2017 IEEE International Conference on Prognostics and Health Management (ICPHM) (p. 88-95).
Issue
Section
Technical Papers