Zero-shot Video Change Detection for Real-life Industrial Applications

##plugins.themes.bootstrap3.article.main##

##plugins.themes.bootstrap3.article.sidebar##

Published Nov 5, 2024
Mahbubul Alam Huimin Zhuge Teresa Gonzalez Ahmed Farahat Song Wang Chetan Gupta

Abstract

Change detection is crucial for various industrial applications. Although image change detection datasets are abundant, the collection of labeled video data is time-consuming, expensive, and cumbersome. This scarcity of labeled data motivates the development of few-shot or zero-shot video change detection techniques which may generalize well to new situations. Existing video change detection methods require large amounts of labeled data, are task-specific, and difficult to generalize. Therefore, in this paper, we propose a zero-shot video change detection algorithm using pre-trained deep learning models and conventional image processing techniques. Our approach identifies matching frames from input videos, adjusts lighting conditions if necessary, and uses an existing object detection model to identify objects in both frames. The method is easily generalizable by making few changes. We evaluate our proposed method on the VDAO dataset collected in a cluttered industrial environment and demonstrate its effectiveness in detecting changes between pairs of videos containing single and multiple objects.

How to Cite

Alam, M., Zhuge, H., Gonzalez, T., Farahat, A., Wang, S., & Gupta, C. (2024). Zero-shot Video Change Detection for Real-life Industrial Applications. Annual Conference of the PHM Society, 16(1). https://doi.org/10.36001/phmconf.2024.v16i1.3903
Abstract 39 | PDF Downloads 44

##plugins.themes.bootstrap3.article.details##

Keywords

Change Detection, Zero-shot Algorithm, Industry Applications, Video data

References
1. Ahmad, F., Shan, Z., Wang, K., You, S., Zhou, B., Gu, S., . . . Xu, Z. (2023). Ez-vcd: Efficient zero-shot video change detection with pretrained visual-language models. arXiv preprint arXiv:2303.01339.


2. Bai, X., Ma, C., Li, Y., Guo, J.-J., Xia, Z., & Guo, H. (2019). Video change detection with fully convolutional networks. Multimedia Tools and Applications, 78(16), 23205–23230.

3. Bruzzone, L., Rizzo, D., Gaddi, L., & Marconcini, M. (2004). A novel approach to the selection of spatially invariant features for change detection in high-resolution images. Pattern Recognition Society, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer, 1, I–I.

4. Celik, T. (2010). Unsupervised change detection in satellite images using principal component analysis and kmeans clustering. IEEE Geoscience and Remote Sensing Letters, 7(4), 739–743.

5. Chen, K., Zhang, B., Yang, Z., Che‘n, Y., Zhong, Y., Li, G., . . . Li, H. (2022). A survey of deep learning-based remote sensing image change detection. In Remote sensing (Vol. 14, p. 4630).

6. Chen, Z., Xu, Y.,Wang, C., Chen, Q., Yang, X., et al. (2018). Real-time video change detection using deep-learning based models. arXiv preprint arXiv:1810.00563.

7. Cholakkal, A. A., Jinek, S., Qi, Y., Jiang, W., Cholakkal, A., Ramakrishnan, S., & Davis, L. S. (2022). Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.

8. Daudt, R., Kleine, R., Glasmachers, T., Ball, L., Rosendahl, P., & Wesemann, L. (2018). Deep transfer learning for efficient image change detection. In arxiv preprint arxiv:1805.12552.

9. Elgammal, A., Harwood, D., & Davis, L. (2000). Nonparametric model for background subtraction. European Conference on Computer Vision, 751–767.

10. Freitas, G., Lopes, N., Jorge, R., Viana, A., Pontes, B., & Ribeiro, R. (2014). An annotated video database for abandoned-object detection in a cluttered environment. In International telecommunications symposium (its), 2014 (pp. 1162–1166).

11. Gan, C., Feng, Y., Blasch, E., Shen, J., Zhang, W., Le, J.- B., . . . others (2017). Deck: Deep event composition knowledge for zero-shot event detection in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2072–2081.

12. Hu, Y., Guo, Y., Xu, Y., Liu, Z., Yuan, Y.-P., et al. (2011). Video change detection based on frame differencing and gaussian mixture modeling. Pattern Recognition Letters, 32(14), 1727–1734.

13. Laptev, I., Weickert, M. H. S., & Rae, M. (2022). Multiscale video sequence matching for near-duplicate detection and localization. IEEE Transactions on Multimedia, 24(2), 1151–1162.

14. Li, F., Zhang, B., Han, X., Chen, K., Zhang, G., & Wu, Y. (2022). Remote sensing image change detection with transformers. In arxiv preprint arxiv:2210.09272.

15. Mahadevan, V., & Vasconcelos, N. (2012). Scene change detection: A survey. Pattern Recognition, 46(1), 398– 408.

16. Malila, W. A. (1980). Change detection in urban areas using multispectral aerial photography. Proceedings of the ESA/JRC Workshop on Remote Sensing for Environmental Monitoring of Urban Areas, 1–8.

17. Mittal, A., & Zisserman, A. (2013). Video change detection using optical flow based descriptors. International Conference on Computer Vision, 455–462.

18. Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2001). Color transfer between images. IEEE Computer Graphics and Applications, 21(5), 34–41.

19. Saha, S., Chaudhury, K., Banerjee, A., & Saha, B. (2013). Change detection in video using local binary pattern. International Journal of Engineering Research & Technology, 2(5), 1462–1467.

20. Singh, A., Harrison, J., & Aggarwal, K. (1989). Image differencing and monitoring urban land-use change. Photogrammetric Engineering and Remote Sensing, 55(10), 1357–1368.

21. Telle, L., Liao, H.-T., Tao, H., Xu, A., Zha, H., Tokmakov, P., . . . others (2023). Lasersam: Scene-aware semantic annotation of 3d point clouds. arXiv preprint arXiv:2303.07930.

22. Teller, S. B., Larsen, K., & Foga, M. (2022). Video matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3), 1443–1456.

23. Yang, M., He, Q., Chen, Y., Tian, Q., & Yu, D. (2020). Video change detection with a two-stream convolutional neural network. Neurocomputing, 403, 341–348.

24. Zhang, B., Guo, H., Chen, K., Li, S., Sun, K., Li, J., . . . Wu, Y. (2022). Maskcd: A remote sensing change detection benchmark with annotation masks. In arxiv preprint arxiv:2212.05316.

25. Zhu, L., Zhan, H., Liu, C., You, F., Qin, R., et al. (2021). Video change detection with transformers. arXiv preprint arXiv:2102.12720.
Section
Technical Research Papers

Most read articles by the same author(s)