A Dynamic Weighted RBF-Based Ensemble for Prediction of Time Series Data from Nuclear Components
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Nov 3, 2020
Jie Liu
Valeria Vitelli
Enrico Zio
Redouane Seraoui
Abstract
In this paper, an ensemble approach is proposed for prediction of time series data based on a Support Vector Regression (SVR) algorithm with RBF loss function. We propose a strategy to build diverse sub-models of the ensemble based on the Feature Vector Selection (FVS) method of Baudat & Anouar (2003), which decreases the computational burden and keeps the generalization performance of the model. A simple but effective strategy is used to calculate the weights of each data point for different sub-models built with RBF-SVR. A real case study on a nuclear power production component is presented. Comparisons with results given by the best single SVR model and a fixed-weights ensemble prove the robustness and accuracy of the proposed ensemble approach.
##plugins.themes.bootstrap3.article.details##
Keywords
Ensemble, feature vector selection, dynamic weights calculation
References
Acar, E., & Rais-Rohani, M. (2009). Ensemble of metamodels with optimized weight factors. Structural and Multidisciplinary Optimization, 37(3), 279-294.
Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1-2), 105-139.
Baudat, G., & Anouar, F. (2003). Feature vector selection and projection using kernels. Neurocomputing, 55(1-2), 21-38.
Bonissone, Piero P., Feng Xue, and Raj Subbu. "Fast meta-models for local fusion of multiple predictive models." Applied Soft Computing 11.2 (2011): 1529-1539.
Brodsky, J. B., Lemmens, H. J., Brock-Utne, J. G., Vierra, M., & Saidman, L. J. (2002). Morbid obesity and tracheal intubation. Anesthesia & Analgesia,94(3), 732-736.
Chen, S., Wang, W., & Van Zuylen, H. (2009). Construct support vector machine ensemble to detect traffic incident. Expert systems with applications, 36(8), 10976-10986.
Fantoni, Paolo F. "A neuro-fuzzy model applied to full range signal validation of PWR nuclear power plant data." INTERNATIONAL JOURNAL OF GENERAL SYSTEM 29.2 (2000): 305-320.
Gao, J. B., Gunn, S. R., Harris, C. J., & Brown, M. (2002). A probabilistic framework for SVM regression and error bar estimation. Machine Learning,46(1-3), 71-89.
Gurram, P., & Kwon, H. (2013). Sparse kernel-based ensemble learning with fully optimized kernel parameters for hyperspectral classification problems.Geoscience and Remote Sensing, IEEE Transactions on, 51(2), 787-802.
Hallbert B. & Thomas K. (2014), Advanced Instrumentation, Information, and Control Systems Technologies Technical Program Plan for 2014, Idaho National Laboratory, INL/EXT-13-28055 Rev. 3, September 2014.
Hu, C., Youn, B. D., Wang, P., & Taek Yoon, J. (2012). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Reliability Engineering & System Safety, 103, 120-135.
Koo, In Soo, and Whan Woo Kim. "The development of reactor coolant pump vibration monitoring and a diagnostic system in the nuclear power plant." ISA transactions 39.3 (2000): 309-316.
Kim, H. C., Pang, S., Je, H. M., Kim, D., & Yang Bang, S. (2003). Constructing support vector machine ensemble. Pattern recognition, 36(12), 2757-2767.
Lee, Terence. "Assessment of safety culture at a nuclear reprocessing plant." Work & Stress 12.3 (1998): 217-237.
Liu, J., Seraoui, R., Vitelli, V., & Zio, E. (2013). Nuclear power plant components condition monitoring by probabilistic support vector machine. Annals of Nuclear Energy, 56, 23-33.
Masry, E. (1996). Multivariate local polynomial regression for time series: uniform strong consistency and rates. Journal of Time Series Analysis, 17(6), 571-599.
Minh, H. Q., Niyogi, P., & Yao, Y. (2006). Mercer’s theorem, feature maps, and smoothing. In Learning theory (pp. 154-168). Springer Berlin Heidelberg.
Muhlbaier, M. D., Topalis, A., & Polikar, R. (2009). Learn. NC: Combining Ensemble of Classifiers With Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes. Neural Networks, IEEE Transactions on, 20(1), 152-168.
Polikar, R. (2006). Ensemble based systems in decision making. Circuits and Systems Magazine, IEEE, 6(3), 21-45.
Quinlan J R. Bagging, boosting, and C4. 5[C]//AAAI/IAAI, Vol. 1. 1996: 725-730.
Razavi-Far, R., Baraldi, P., & Zio, E. (2012). Dynamic Weighting Ensembles for Incremental Learning and Diagnosing New Concept Class Faults in Nuclear Power Systems. Nuclear Science, IEEE Transactions on, 59(5), 2520-2530.
Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and computing, 14(3), 199-222.
Sollich, P. (1999). Probabilistic interpretations and Bayesian methods for support vector machines. In Artificial Neural Networks, 1999. ICANN 99. Ninth International Conference on (Conf. Publ. No. 470) (Vol. 1, pp. 91-96). IET.
Valentini, G., & Dietterich, T. G. (2003, August). Low bias bagged support vector machines. In ICML (pp. 752-759).
Yang, X., Yuan, B., & Liu, W. (2009, November). Dynamic Weighting Ensembles for incremental learning. In Proc. of IEEE conference in pattern recognition (pp. 1-5).
Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1-2), 105-139.
Baudat, G., & Anouar, F. (2003). Feature vector selection and projection using kernels. Neurocomputing, 55(1-2), 21-38.
Bonissone, Piero P., Feng Xue, and Raj Subbu. "Fast meta-models for local fusion of multiple predictive models." Applied Soft Computing 11.2 (2011): 1529-1539.
Brodsky, J. B., Lemmens, H. J., Brock-Utne, J. G., Vierra, M., & Saidman, L. J. (2002). Morbid obesity and tracheal intubation. Anesthesia & Analgesia,94(3), 732-736.
Chen, S., Wang, W., & Van Zuylen, H. (2009). Construct support vector machine ensemble to detect traffic incident. Expert systems with applications, 36(8), 10976-10986.
Fantoni, Paolo F. "A neuro-fuzzy model applied to full range signal validation of PWR nuclear power plant data." INTERNATIONAL JOURNAL OF GENERAL SYSTEM 29.2 (2000): 305-320.
Gao, J. B., Gunn, S. R., Harris, C. J., & Brown, M. (2002). A probabilistic framework for SVM regression and error bar estimation. Machine Learning,46(1-3), 71-89.
Gurram, P., & Kwon, H. (2013). Sparse kernel-based ensemble learning with fully optimized kernel parameters for hyperspectral classification problems.Geoscience and Remote Sensing, IEEE Transactions on, 51(2), 787-802.
Hallbert B. & Thomas K. (2014), Advanced Instrumentation, Information, and Control Systems Technologies Technical Program Plan for 2014, Idaho National Laboratory, INL/EXT-13-28055 Rev. 3, September 2014.
Hu, C., Youn, B. D., Wang, P., & Taek Yoon, J. (2012). Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life. Reliability Engineering & System Safety, 103, 120-135.
Koo, In Soo, and Whan Woo Kim. "The development of reactor coolant pump vibration monitoring and a diagnostic system in the nuclear power plant." ISA transactions 39.3 (2000): 309-316.
Kim, H. C., Pang, S., Je, H. M., Kim, D., & Yang Bang, S. (2003). Constructing support vector machine ensemble. Pattern recognition, 36(12), 2757-2767.
Lee, Terence. "Assessment of safety culture at a nuclear reprocessing plant." Work & Stress 12.3 (1998): 217-237.
Liu, J., Seraoui, R., Vitelli, V., & Zio, E. (2013). Nuclear power plant components condition monitoring by probabilistic support vector machine. Annals of Nuclear Energy, 56, 23-33.
Masry, E. (1996). Multivariate local polynomial regression for time series: uniform strong consistency and rates. Journal of Time Series Analysis, 17(6), 571-599.
Minh, H. Q., Niyogi, P., & Yao, Y. (2006). Mercer’s theorem, feature maps, and smoothing. In Learning theory (pp. 154-168). Springer Berlin Heidelberg.
Muhlbaier, M. D., Topalis, A., & Polikar, R. (2009). Learn. NC: Combining Ensemble of Classifiers With Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes. Neural Networks, IEEE Transactions on, 20(1), 152-168.
Polikar, R. (2006). Ensemble based systems in decision making. Circuits and Systems Magazine, IEEE, 6(3), 21-45.
Quinlan J R. Bagging, boosting, and C4. 5[C]//AAAI/IAAI, Vol. 1. 1996: 725-730.
Razavi-Far, R., Baraldi, P., & Zio, E. (2012). Dynamic Weighting Ensembles for Incremental Learning and Diagnosing New Concept Class Faults in Nuclear Power Systems. Nuclear Science, IEEE Transactions on, 59(5), 2520-2530.
Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and computing, 14(3), 199-222.
Sollich, P. (1999). Probabilistic interpretations and Bayesian methods for support vector machines. In Artificial Neural Networks, 1999. ICANN 99. Ninth International Conference on (Conf. Publ. No. 470) (Vol. 1, pp. 91-96). IET.
Valentini, G., & Dietterich, T. G. (2003, August). Low bias bagged support vector machines. In ICML (pp. 752-759).
Yang, X., Yuan, B., & Liu, W. (2009, November). Dynamic Weighting Ensembles for incremental learning. In Proc. of IEEE conference in pattern recognition (pp. 1-5).
Section
Technical Papers