Zero-shot Video Change Detection for Real-life Industrial Applications
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
Change detection is crucial for various industrial applications. Although image change detection datasets are abundant, the collection of labeled video data is time-consuming, expensive, and cumbersome. This scarcity of labeled data motivates the development of few-shot or zero-shot video change detection techniques which may generalize well to new situations. Existing video change detection methods require large amounts of labeled data, are task-specific, and difficult to generalize. Therefore, in this paper, we propose a zero-shot video change detection algorithm using pre-trained deep learning models and conventional image processing techniques. Our approach identifies matching frames from input videos, adjusts lighting conditions if necessary, and uses an existing object detection model to identify objects in both frames. The method is easily generalizable by making few changes. We evaluate our proposed method on the VDAO dataset collected in a cluttered industrial environment and demonstrate its effectiveness in detecting changes between pairs of videos containing single and multiple objects.
How to Cite
##plugins.themes.bootstrap3.article.details##
Change Detection, Zero-shot Algorithm, Industry Applications, Video data
2. Bai, X., Ma, C., Li, Y., Guo, J.-J., Xia, Z., & Guo, H. (2019). Video change detection with fully convolutional networks. Multimedia Tools and Applications, 78(16), 23205–23230.
3. Bruzzone, L., Rizzo, D., Gaddi, L., & Marconcini, M. (2004). A novel approach to the selection of spatially invariant features for change detection in high-resolution images. Pattern Recognition Society, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer, 1, I–I.
4. Celik, T. (2010). Unsupervised change detection in satellite images using principal component analysis and kmeans clustering. IEEE Geoscience and Remote Sensing Letters, 7(4), 739–743.
5. Chen, K., Zhang, B., Yang, Z., Che‘n, Y., Zhong, Y., Li, G., . . . Li, H. (2022). A survey of deep learning-based remote sensing image change detection. In Remote sensing (Vol. 14, p. 4630).
6. Chen, Z., Xu, Y.,Wang, C., Chen, Q., Yang, X., et al. (2018). Real-time video change detection using deep-learning based models. arXiv preprint arXiv:1810.00563.
7. Cholakkal, A. A., Jinek, S., Qi, Y., Jiang, W., Cholakkal, A., Ramakrishnan, S., & Davis, L. S. (2022). Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.
8. Daudt, R., Kleine, R., Glasmachers, T., Ball, L., Rosendahl, P., & Wesemann, L. (2018). Deep transfer learning for efficient image change detection. In arxiv preprint arxiv:1805.12552.
9. Elgammal, A., Harwood, D., & Davis, L. (2000). Nonparametric model for background subtraction. European Conference on Computer Vision, 751–767.
10. Freitas, G., Lopes, N., Jorge, R., Viana, A., Pontes, B., & Ribeiro, R. (2014). An annotated video database for abandoned-object detection in a cluttered environment. In International telecommunications symposium (its), 2014 (pp. 1162–1166).
11. Gan, C., Feng, Y., Blasch, E., Shen, J., Zhang, W., Le, J.- B., . . . others (2017). Deck: Deep event composition knowledge for zero-shot event detection in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2072–2081.
12. Hu, Y., Guo, Y., Xu, Y., Liu, Z., Yuan, Y.-P., et al. (2011). Video change detection based on frame differencing and gaussian mixture modeling. Pattern Recognition Letters, 32(14), 1727–1734.
13. Laptev, I., Weickert, M. H. S., & Rae, M. (2022). Multiscale video sequence matching for near-duplicate detection and localization. IEEE Transactions on Multimedia, 24(2), 1151–1162.
14. Li, F., Zhang, B., Han, X., Chen, K., Zhang, G., & Wu, Y. (2022). Remote sensing image change detection with transformers. In arxiv preprint arxiv:2210.09272.
15. Mahadevan, V., & Vasconcelos, N. (2012). Scene change detection: A survey. Pattern Recognition, 46(1), 398– 408.
16. Malila, W. A. (1980). Change detection in urban areas using multispectral aerial photography. Proceedings of the ESA/JRC Workshop on Remote Sensing for Environmental Monitoring of Urban Areas, 1–8.
17. Mittal, A., & Zisserman, A. (2013). Video change detection using optical flow based descriptors. International Conference on Computer Vision, 455–462.
18. Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2001). Color transfer between images. IEEE Computer Graphics and Applications, 21(5), 34–41.
19. Saha, S., Chaudhury, K., Banerjee, A., & Saha, B. (2013). Change detection in video using local binary pattern. International Journal of Engineering Research & Technology, 2(5), 1462–1467.
20. Singh, A., Harrison, J., & Aggarwal, K. (1989). Image differencing and monitoring urban land-use change. Photogrammetric Engineering and Remote Sensing, 55(10), 1357–1368.
21. Telle, L., Liao, H.-T., Tao, H., Xu, A., Zha, H., Tokmakov, P., . . . others (2023). Lasersam: Scene-aware semantic annotation of 3d point clouds. arXiv preprint arXiv:2303.07930.
22. Teller, S. B., Larsen, K., & Foga, M. (2022). Video matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3), 1443–1456.
23. Yang, M., He, Q., Chen, Y., Tian, Q., & Yu, D. (2020). Video change detection with a two-stream convolutional neural network. Neurocomputing, 403, 341–348.
24. Zhang, B., Guo, H., Chen, K., Li, S., Sun, K., Li, J., . . . Wu, Y. (2022). Maskcd: A remote sensing change detection benchmark with annotation masks. In arxiv preprint arxiv:2212.05316.
25. Zhu, L., Zhan, H., Liu, C., You, F., Qin, R., et al. (2021). Video change detection with transformers. arXiv preprint arXiv:2102.12720.
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.