Enhancing Wagon Fault Detection Using Acoustic Signals and Deep Transfer Learning From Scratch to Full Fine-Tuning
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
This paper proposes a deep transfer learning approach for detecting brake fluid leakage in gondola wagons using acoustic signals. Gondola wagons, also known as railroad gondolas, gondola cars, and open wagons are typically used for transporting dry cargo and rely on pneumatic brake systems that depend on compressed air components for effective braking. The study investigates three transfer-learning strategies—From-Scratch, Partial Fine-Tuning, and Full Fine-Tuning—to identify compressed air leakage based on sound emissions, mirroring the process performed by human wagon inspection professionals. Training data consists of waveform audio signals captured during real railcar inspections. In the proposed model, each audio file is processed in the time-frequency domain to obtain a melspectrogram, which is then used as input to pre-trained deep convolutional neural networks. Results demonstrate strong performance, achieving accuracy above 94%. The main scientific contribution lies in applying established deep learning techniques to a specific and underexplored industrial railway context, together with the development and validation of a real-world dataset collected under authentic operating conditions. These findings demonstrate the feasibility and practical effectiveness of the proposed approach for pneumatic brake leakage detection in operational environments.
##plugins.themes.bootstrap3.article.details##
Railway, open wagons, intelligent fault detection, air braking system, deep transfer learning
ANTF, Associacao Nacional dos Transportadores Ferroviários. (2023). O setor ferroviário de carga brasileiro. https ://www.antf.org .br/o-setor-ferroviario-de-carga-brasileiro. (Accessed: 2024-10-05)
Arunkumar, K. M., & Manjunath, T. C. (2016). A brief review/survey of vibration signal analysis in time domain. International Journal of Electronics and Communication Engineering, 3(3), 12-15. Retrieved from https://doi .org/10 .14445/23488549/IJECE-V3I3P104 doi: 10.14445/ 23488549/IJECE-V3I3P104
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. doi: 10.1613/jair.953
Chen, S. (2022). Deep learning-based sound identification. http://ens.ewi.tudelft.nl/islandora.
Chen, Y., Guo, Q., Liang, X., Wang, J., & Qian, Y. (2019). Environmental sound classification with dilated convolutions. Applied Acoustics, 148, 123–132.
Chen, Z., Xia, J., Li, J., Chen, J., Huang, R., Jin, G., & Li, W. (2023). Generalized open-set domain adaptation in mechanical fault diagnosis using multiple metric weighting learning network. Advanced Engineering Informatics, 56, 101781. doi: 10.1016/j.aei.2023.101781
Cheng, C., Qiao, X., Luo, H., Teng, W., Gao, M., Zhang, B., & Yin, X. (2019). A semi-quantitative information based fault diagnosis method for the running gears system of high-speed trains. IEEE Access, 7, 38168-38178. Retrieved from https://doi.org/10.1109/ACCESS.2019.2901022 doi: 10.1109/ACCESS.2019.2901022
Deng, J., Dong, W., Socher, R., Li, L., Kai, L., & Li, F. (2009). Imagenet: A large-scale hierarchical image database. In 2009 ieee conference on computer vision and pattern recognition (p. 248-255). doi: 10.1109/CVPR.2009.5206848
Ge, X., Chen, Q., Ling, L., Zhai, W., & Wang, K. (2023). An approach for simulating the air brake system of long freight trains based on fluid dynamics. Railway Engineering Science, 31(2), 122–134. Retrieved from https://link.springer.com/article/10.1007/s40534-022-00291-0 doi: 10.1007/s40534-022-00291-0
Giorgi, B. D., Levy, M., & Apple, R. S. (2022). Mel spectrogram inversion with stable pitch. Apple Machine Learning Research. Retrieved from https://machinelearning .apple .com/research/mel-spectrogram
Godbole, V., Dahl, G. E., Gilmer, J., Shallue, C. J., & Nado, Z. (2022). Deep learning tuning playbook. https://github .com/google -research/tuning playbook.
Hashmi, M., Ibrahim, M., Bajwa, I., Siddiqui, H.-U.-R., Rustam, F., Lee, E., & Ashraf, I. (2022). Railway track inspection using deep learning based on audio to spectrogram conversion: An on-the-fly approach. Sensors, 22(5), 1983. Retrieved from https://www.mdpi.com/1424-8220/22/5/1983 doi:10.3390/s22051983
Hoang, N. T., & Kim, D. (2022). Supervised contrastive resnet and transfer learning for the in-vehicle intrusion detection system. Expert Systems with Applications, 209, 118390. doi: 10.1016/j.eswa.2022.118390
Hu, R., Zhang, M., Meng, X., & Kang, Z. (2022). Deep subdomain generalisation network for health monitoring of high-speed train brake pads. Engineering Applications of Artificial Intelligence, 113, 104896. doi: 10.1016/j.engappai.2022.104896
Jin, Y., Xie, G., Li, Y., Zhang, X., Han, N., Shangguan, A., & Chen, W. (2021). Fault diagnosis of brake train based on multi-sensor data fusion. Sensors, 21(13), 4370. Retrieved from https://doi .org/10 .3390/s21134370 doi: 10.3390/s21134370
Karabacak, Y. E., Özmen, N. G., & Gumusel, L. (2022). Intelligent worm gearbox fault diagnosis under various working conditions using vibration, sound and thermal features. Applied Acoustics, 186, 108463. Retrieved from https://www.sciencedirect.com/science/article/pii/S0003682X21005570 doi: 10.1016/j.apacoust.2021.108463
Kensert, A., Harrison, P. J., & Spjuth, O. (2018, October). Transfer learning with deep convolutional neural networks for classifying cellular morphological changes. In Uppsala university faculty of pharmacy 50th anniversary. Uppsala University. Retrieved from https://pharmb .io/poster/2018 -transfer-cnn/ doi: 10.6084/m9.figshare.9747029.v1
Kulkarni, R., Qazizadeh, A., & Berg, M. (2023). Unsupervised rail vehicle running instability detection algorithm for passenger trains (ivrida). Measurement, 217, 112372. Retrieved from https://www.sciencedirect.com/science/article/pii/S026322412300122X doi: 10.1016/j.measurement.2023.112372
Liu, H., Peng, J., Gao, D., Yang, Y., Fan, Y., Hu, C., & Zhang, X. (2020). A hybrid data-fusion estimate method for health status of train braking system. In Proceedings of the 2020 ieee international conference on systems, man, and cybernetics (smc) (pp. 2130–2135). Toronto, ON, Canada: IEEE. doi: 10.1109/SMC42913.2020.9281152
Luz, J. S., Oliveira, M. C., Aráujo, F. H. D., & Magalhaes, D. M. V. (2021). Ensemble of handcrafted and deep features for urban sound classification. Applied Acoustics, 175, 107819. Retrieved from https://www.sciencedirect .com/science/article/pii/S0003682X20309245 doi: 10.1016/j.apacoust.2020.107819
Ma, S., Leng, J., Chen, Z., Li, B., Li, X., Zhang, D., . . . Liu, Q. (2024). A novel weakly supervised adversarial network for thermal error modeling of electric spindles with scarce samples. Expert Systems with Applications, 238, 122065. doi: 10.1016/j.eswa.2023.122065
McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and music signal analysis in python. In In proceedings of the 14th python in science conference (pp. 18–25). SciPy.
Padha, A., & Sahoo, A. (2023). Qclr: Quantum-lstm contrastive learning framework for continuous mental health monitoring. Expert Systems with Applications, 226, 120761. doi: 10.1016/j.eswa.2023.120761
Patil, S. S., Pardeshi, S. S., & Patange, A. D. (2023). Health monitoring of milling tool inserts using cnn architectures trained by vibration spectrograms. CMES- Computer Modeling in Engineering and Sciences, 139(2), 487-500. Retrieved from https://www.techscience.com/cmes/v139n2/49963 doi: 10.32604/cmes.2023.022924
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, E. (2011). Scikitlearn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. Retrieved from http://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf
Pfaff, R., Enning, M., & Sutter, S. (2024). A risk-based approach to automatic brake tests for rail freight service: incident analysis and realisation concept. Journal of Rail Transport, 15(2), 123-145.
Pugi, L., Rosano, G., Viviani, R., Cabrucci, L., & Bocciolini, L. (2023). Modeling, testing and validation of the vibrational behavior of a dynamometric test rig for railway braking systems. World Journal of Engineering, 1–18. Retrieved from https://www.emerald .com/insight/content/doi/10.1108/WJE-04-2023-0073/full/html doi: 10.1108/WJE-04-2023-0073
Rauber, T., Loca, A. L. S., Boldt, F. A., Rodrigues, A. L., & Varejao, F. M. (2021). An experimental methodology to evaluate machine learning methods for fault diagnosis based on vibration signals. Experts Systems with Applications, 167, 114022. doi: https://doi.org/10.1016/j.eswa.2020.114022
Reis, J. L., & Varejao, F. M. (2023). Deteccao de vazamentos de fluidos de freios a ar em vagoes do tipo gôndola através do sinal acústico: Um modelo de classificacao de falhas. Revista Foco, 16(5), e1885. Retrieved from https://doi.org/10.54751/revistafoco.v16n5-072 doi: 10.54751/revistafoco.v16n5-072
Ribeiro, R. P., Pereira, P., & Gama, J. (2016). Sequential anomalies: a study in the railway industry. Machine Learning, 105(1), 127–153. doi: 10.1007/s10994-016-5584-0
Rodrigues, P. L. C., Jutten, C., & Congedo, M. (2023). Riemannian procrustes analysis: Transfer learning for braincomputer interfaces. IEEE Transactions on Biomedical Engineering, 70(9), 2715-2725. doi: 10.1109/TBME.2023.3235513
Sakellariou, J. S., Petsounis, K. A., & Fassois, S. D. (2014). Vibration-based fault diagnosis for railway vehicle suspensions via a functional model-based method: A feasibility study. Journal of Mechanical Science and Technology, 28(11), 4443–4451. doi: 10.1007/s12206-014-1019-2
Sangeetha, P., & Hemamalini, S. (2017). Dyadic wavelet transform-based acoustic signal analysis for torque prediction of a three-phase induction motor. IET Signal Processing, 11(5), 604–612. doi: 10.1049/iet-spr.2016.0165
Tagawa, Y., Maskeliünas, R., & Damasevicius, R. (2021). Acoustic anomaly detection of mechanical failures in noisy real-life factory environments. Electronics (Switzerland), 10(19), 2329. doi: 10 .3390/electronics10192329
Umesh, S., Cohen, L., & Nelson, D. (1999). Fitting the mel scale. In Icassp, ieee international conference on acoustics, speech and signal processing - proceedings (pp. 217–220). IEEE.
Ustubioglu, A., Ustubioglu, B., & Ulutas, G. (2023). Mel spectrogram-based audio forgery detection using cnn. Signal, Image and Video Processing, 17(5), 1915-1924. doi: 10.1007/s11760-022-02274-y
Vrbancic, G., & Podgorelec, V. (2020). Transfer learning with adaptive fine-tuning. IEEE Access, 8, 196197–196211. Retrieved from https://doi.org/10.1109/ACCESS.2020.3034343 doi: 10.1109/ACCESS.2020.3034343
Wang, X., Lu, Z., Wei, J., & Zhang, Y. (2019). Fault diagnosis for rail vehicle axle-box bearings based on energy feature reconstruction and composite multiscale permutation entropy. Entropy, 21(9), 865. Retrieved from https://doi.org/10.3390/e21090865 doi: 10.3390/e21090865
Wood, L., Tan, Z., Stenbit, I., Bischof, J., Zhu, S., & Chollet, F. (2022). Kerascv. https://github.com/keras-team/keras-cv.
Wu, J. D., & Chan, J. J. (2009). Faulted gear identification of a rotating machinery based on wavelet transform and artificial neural network. Expert Systems with Applications, 36(5), 8862–8875. doi: 10.1016/j.eswa.2008.11.035
Xiong, L., Liu, Z., & Niu, G. (2021). Failure analysis of electromagnetic-pneumatic valve in bogie-based braking control system. In 2021 global reliability and prognostics and health management, phm-nanjing 2021.
Xu, M., & Yao, H. (2023). Fault diagnosis method of wheelset based on eemd-mpe and support vector machine optimized by quantum-behaved particle swarm algorithm. Measurement, 216, 112923. doi: 10.1016/j.measurement.2023.112923
Yao, Y., Wang, H., Li, S., Liu, Z., Gui, G., Dan, Y., & Hu, J. (2018). End-to-end convolutional neural network model for gear fault diagnosis based on sound signals. Applied Sciences (Switzerland), 8(9), 1584. doi: 10.3390/app8091584
Ye, Y., Zhang, J., & Liang, H. (2020). An acoustic-based recognition algorithm for the unreleased braking of railway wagons in marshalling yards. IEEE Access, 8, 78038–78048. doi: 10.1109/ACCESS.2020.3006003
Yusof, M. F. M., Kamaruzaman, M. A., Ishak, M., & Ghazali, M. F. (2017). Porosity detection by analyzing arc sound signal acquired during the welding process of gas pipeline steel. International Journal of Advanced Manufacturing Technology, 89(9-12), 3661-3670. doi: 10.1007/s00170-016-9343-4
Zanelli, F., Mauri, M., Castelli-Dezza, F., Sabbioni, E., Tarsitano, D., & Debattisti, N. (2022). Energy autonomous wireless sensor nodes for freight train braking systems monitoring. Sensors, 22(5), 1876. Retrieved from https://doi.org/10.3390/s22051876 doi: 10.3390/s22051876
Zhang, M., Liu, Z., & Dang, X. (2021). Fault diagnosis on train brake system based on multi-dimensional feature fusion and gbdt enhanced classification. Sensors, 21(13), 4370. doi: 10.3390/s21134370
Zhang, T., Peng, P., Tang, X., Yan, R., & Deng, R. (2023). Cme-epc: A coarse-mechanism embedded error prediction and compensation framework for robot multicondition tasks. Robotics and Computer-Integrated Manufacturing, 78, 102436. doi: 10.1016/j.rcim.2022.102436
Zhou, D., Ji, H., He, X., & Shang, J. (2018). Fault detection and isolation of the brake cylinder system for electric multiple units. IEEE Transactions on Control Systems Technology, 26(5), 1744–1757. doi: 10.1109/TCST.2018.2794716
Zhou, Q., Shan, J., Ding, W., Wang, C., Yuan, S., Sun, F., . . . Fang, B. (2021). Cough recognition based on melspectrogram and convolutional neural network. Frontiers in Robotics and AI, 8, 580080. doi: 10.3389/frobt.2021.580080
Zuo, J., Ding, J., & Feng, F. (2019). Latent leakage fault identification and diagnosis based on multi-source information
fusion method for key pneumatic units in chinese standard electric multiple units (emu) braking system. Applied Sciences, 9(2), 300. doi: 10.3390/app9020300