Temporal Learning in Video Data Using Deep Learning and Gaussian Processes
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
This paper presents an approach for data-driven modeling of hidden, stationary temporal dynamics in sequential images or videos using deep learning and Bayesian non-parametric techniques. In particular, a deep Convolutional Neural Network (CNN) is used to extract spatial features in an unsupervised fashion from individual images and then, a Gaussian process is used to model the temporal dynamics of the spatial features extracted by the deep CNN. By decomposing the spatial and temporal components and utilizing the strengths of deep learning and Gaussian processes for the respective sub-problems, we are able to construct a model that is able to capture complex spatio-temporal phenomena while using relatively small number of free parameters. The proposed approach is tested on high-speed grey-scale video data obtained of combustion flames in a swirl-stabilized combustor, where certain protocols are used to induce instability in combustion process. The proposed approach is then used to detect and predict the transition of the combustion process from stable to unstable regime. It is demonstrated that the proposed approach is able to detect unstable flame conditions using very few frames from high-speed video. This is useful as early detection of unstable combustion can lead to better control strategies to mitigate instability. Results from the proposed approach are compared and contrasted with several baselines and recent work in this area. The performance of the proposed approach is found to be significantly better in terms of detection accuracy, model complexity and lead-time to detection.
##plugins.themes.bootstrap3.article.details##
Gaussian Processes, deep learning, Combustion instability
Banaszuk, A., Mehta, P. G., Jacobson, C. A., & Khibnik, A. I. (2006). Limits of achievable performance of controlled combustion processes. Control Systems Technology, IEEE Transactions on, 14(5), 881–895.
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(8), 1798–1828.
Berkooz, G., Holmes, P., & Lumley, J. L. (1993). The proper orthogonal decomposition in the analysis of turbulent flows. Annual review of fluid mechanics, 25(1), 539–575.
Candel, S., Durox, D., Schuller, T., Bourgouin, J.-F., & Moeck, J. P. (2014). Dynamics of swirling flames. Annual review of fluid mechanics, 46, 147–173.
Darema, F. (2005). Dynamic data driven applications systems: New capabilities for application simulations and measurements. In International conference on computational science (pp. 610–615).
Hauser, M., Li, Y., Li, J., & Ray, A. (2016). Real-time combustion state identification via image processing: A dynamic data-driven approach. In 2016 American Control Conference (ACC) (p. 3316-3321).
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.
Huang, Y., & Yang, V. (2009). Dynamics and stability of lean-premixed swirl-stabilized combustion. Progress in Energy and Combustion Science, 35(4), 293–364.
Jha, D., Virani, N., & Ray, A. (2016). Markov moldeing of time-series data via spectral analysis. In 1st International Conference on InfoSymbiotics/DDDAS, Hartford, CT.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097–1105).
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
Le Cun Y., B. B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1990). Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems.
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning (pp. 609–616).
Moeck, J. P., Bourgouin, J.-F., Durox, D., Schuller, T.,&Candel, S. (2012). Nonlinear interaction between a precessing vortex core and acoustic oscillations in a turbulent swirling flame. Combustion and Flame, 159(8), 2650–2668.
O’Connor, J., Acharya, V., & Lieuwen, T. (2015). Transverse combustion instabilities: Acoustic, fluid mechanic, and flame processes. Progress in Energy and Combustion Science, 49, 1–39.
Rasmussen, C. E., & Williams, C. (2006). Gaussian processes for machine learning. The MIT Press, Cambridge, MA, USA.
Sarkar, S., Jha, D., Lore, K., Sarkar, S., & Ray, A. (2016). Multimodal spatiotemporal information fusion using neural-symbolic modeling for early detection of combustion instabilities. In 2016 American Control Conference (ACC) (p. 4918-4923).
Sarkar, S., Lore, K. G., & Sarkar, S. (2015). Early detection of combustion instability by neural-symbolic analysis on hi-speed video. In Workshop on cognitive computation: Integrating neural and symbolic approaches (coco@ nips 2015), Montreal, Canada.
Sarkar, S., Lore, K. G., Sarkar, S., Ramanan, V., Chakravarthy, S. R., Phoha, S., & Ray, A. (2015). Early detection of combustion instability from hi-speed flame images via deep learning and symbolic time series analysis.
Schmid, P. J. (2010). Dynamic mode decomposition of numerical and experimental data. Journal of fluid mechanics, 656, 5–28.
S´e, Ducruix, b., Schuller, T., Durox, D., S´e, & Candel, b. (2003). Combustion dynamics and instabilities: Elementary coupling and driving mechanisms. Journal of Propulsion and Power, 19(5), 722–734.
Virani, N., Jha, D., & Ray, A. (2016). Sequential hypothesis tests using markov models of time-series data. In 1st ACM SIGKDD workshop on machine learning for prognostics and health management, San Fransisco, CA.