Sound-Dr: Reliable Sound Dataset and Baseline Artificial Intelligence System for Respiratory Illnesses
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Sep 4, 2023
Van Truong Hoang
Quang Nguyen
Quoc Cuong Nguyen
Xuan Phong Nguyen
Hoang Nguyen
Abstract
As the burden of respiratory diseases continues to fall on society worldwide, this paper proposes a high-quality and reliable dataset of human sounds for studying respiratory illnesses, including pneumonia and COVID-19. It consists of coughing, mouth breathing, and nose breathing sounds together with metadata on related clinical characteristics. We also develop a proof-of-concept system for establishing baselines and benchmarking against multiple datasets, such as Coswara and COUGHVID. Our comprehensive experiments
show that the Sound-Dr dataset has richer features, better performance, and is more robust to dataset shifts in various machine learning tasks. It is promising for a wide range of real-time applications on mobile devices. The proposed dataset and system will serve as practical tools to support healthcare professionals in diagnosing respiratory disorders. The dataset and code are publicly available here: https://github.com/ReML-AI/Sound-Dr/.
##plugins.themes.bootstrap3.article.details##
Keywords
Deep Learning, Anomaly, Respiratory, Baseline, Healthcare
References
Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. In The 25th acm sigkdd international conference on knowledge discovery and data mining (p. 2623–2631). doi: 10.1145/3292500 .3330701
Amiriparian, S., Gerczuk, M., Ottl, S., Cummins, N., Freitag, M., Pugachevskiy, S., . . . Schuller, B. (2017, August). Snore sound classification using imagebased deep spectrum features. In International speech communication association (interspeech) (pp. 3512– 3516).
Bradley, A. P. (1997). The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145-1159. doi: 10.1016/ S0031-3203(96)00142-2
Brown, C., Chauhan, J., Grammenos, A., Han, J., Hasthanasombat, A., Spathis, D., . . . Mascolo, C. (2020). Exploring automatic diagnosis of covid-19 from crowdsourced respiratory sound data. In Proceedings of the 26th acm sigkdd international conference on knowledge discovery and data mining (p. 3474–3484). doi: 10.1145/3394486.3412865
Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Acm sigkdd international conference on knowledge discovery and data mining (p. 785–794). doi: 10.1145/2939672.2939785
Deb, S., Warule, P., Nair, A., Sultan, H., Dash, R., & Krajewski, J. (2022, 9). Detection of common cold from speech signals using deep neural network. Circuits, Systems, and Signal Processing. doi: 10.1007/s00034-022-02189-y
Eyben, F., Wollmer, M., & Schuller, B. (2010, 01). opensmile – the munich versatile and fast open-source audio feature extractor. ACM Multimedia 2010 International Conference, 1459-1462. doi: 10.1145/1873951.1874246
FPT. (1999). Fpt softwave company limited. Retrieved from https://www.fpt-software.com ([Online; accessed 30-04-2023])
Friedman, J. (2000, 11). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29. doi: 10.1214/aos/1013203451
Hoang, T., Pham, L., Ngo, D., & Nguyen, H. D. (2022). A cough-based deep learning framework for detecting covid-19. In The 44th ieee engineering in medicine and biology society (embc) (p. 3422-3425). doi: 10.1109/ EMBC48229.2022.9871179
Islam, R., Abdel-Raheem, E., & Tarique, M. (2022). A novel pathological voice identification technique through simulated cochlear implant processing systems. Applied Sciences, 12(5). doi: 10.3390/app12052398
Kreiman, J., Gerratt, B. R., & Precoda, K. (1990). Listener experience and perception of voice quality. Journal of Speech, Language, and Hearing Research, 33(1), 103– 115.
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. In Eighth ieee international conference on data mining (p. 413-422). doi: 10.1109/ICDM.2008.17
McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference (Vol. 8, pp. 18–25).
Mo, A., Gui, E., & Fletcher, R. R. (2022). Use of voluntary cough sounds and deep learning for pulmonary disease screening in low-resource areas. In Ieee global humanitarian technology conference (ghtc) (p. 242-249). doi: 10.1109/GHTC55712.2022.9911027
NYU Breathing Sounds for COVID-19. (2020). https:// breatheforscience.com/. ([Online; accessed 30-March-2023])
Orlandic, L., Teijeiro, T., & Atienza, D. (2021). The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms. Springer Science and Business Media LLC, 8(1). doi: 10.1038/s41597-021-00937-4
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Peplinski, J., Shor, J., Joglekar, S., Garrison, J., & Patel, S. (2021). FRILL: A Non-Semantic Speech Embedding for Mobile Devices. In International speech communication association (interspeech) (pp. 1204–1208). doi: 10.21437/Interspeech.2021-2070
Pham, L., Ngo, D., Tran, K., Hoang, T., Schindler, A., & McLoughlin, I. (2022). An Ensemble of Deep Learning Frameworks for Predicting Respiratory Anomalies. In Ieee engineering in medicine & biology society (embc) (p. 4595-4598). doi: 10.1109/EMBC48229 .2022.9871440
Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D. (2009). Dataset shift in machine learning. The MIT Press.
Rabanser, S., Gunnemann, S., & Lipton, Z. C. (2019). Failing loudly: An empirical study of methods for detecting dataset shift. Curran Associates Inc.
Rocha, B. M., Filos, D., Mendes, L., Serbes, G., Ulukaya, S., Kahya, Y. P., . . . de Carvalho, P. (2019, mar). An open access database for the evaluation of respiratory sound classification algorithms. Physiological Measurement, 40(3), 035001. doi: 10.1088/1361-6579/ab03ea
Sakkatos, P., Barney, A., Bruton, A., Haitchi, H. M., Kurukulaaratchy, R. J., & Thackray, D. (2019). Quantified breathing patterns can be used as a physiological marker to monitor asthma. European Respiratory Journal, 54(suppl 63). doi: 10.1183/13993003.congress -2019.PA5038
Sasaki, Y. (2007, 01). The truth of the f-measure. Teach Tutor Mater.
Schmitt, M., & Schuller, B. (2017). openxbow – introducing the passau open-source crossmodal bag-ofwords toolkit. Journal of Machine Learning Research, 18(96), 1–5.
Sharma, N., Krishnan, P., Kumar, R., Ramoji, S., Chetupalli, S. R., R., N., . . . Ganapathy, S. (2020). Coswara — A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis. In International speech communication association (interspeech) (pp. 4811–4815). doi: 10.21437/Interspeech.2020-2768
Sharma, N. K., Chetupalli, S. R., Bhattacharya, D., Dutta, D., Mote, P., & Ganapathy, S. (2022). The second dicova challenge: Dataset and performance analysis for diagnosis of covid-19 using acoustics. In Ieee international conference on acoustics, speech and signal processing (icassp) (p. 556-560). doi: 10.1109/ ICASSP43922.2022.9747188
Shor, J., Jansen, A., Maor, R., Lang, O., Tuval, O., de Chaumont Quitry, F., . . . Haviv, Y. (2020). Towards Learning a Universal Non-Semantic Representation of Speech. In International speech communication association (interspeech) (pp. 140–144). doi: 10.21437/ Interspeech.2020-1242
Song, I. (2015). Diagnosis of pneumonia from sounds collected using low cost cell phones. In International joint conference on neural networks (ijcnn) (p. 1-8). doi: 10.1109/IJCNN.2015.7280317
Woolcock institute of medical research vietnam. (1981). Retrieved from https://www.woolcockvietnam .org ([Online; accessed 30-04-2023])
Yang, Y., Yuan, Y., Zhang, G., Hao Wang, Y.-C. C., Liu, Y., Tarolli, C. G., . . . Katabi, D. (2022, august). Artificial intelligence-enabled detection and assessment of parkinson’s disease using nocturnal breathing signals. Nature Medicine. doi: 10.1038/s41591-022-01932-x
Zhao, Y., & Hryniewicki, M. K. (2018). Xgbod: Improving supervised outlier detection with unsupervised representation learning. In International joint conference on neural networks (ijcnn) (p. 1-8). doi: 10.1109/ IJCNN.2018.8489605
Zhao, Y., Nasrullah, Z., & Li, Z. (2019). Pyod: A python toolbox for scalable outlier detection. Journal of Machine Learning Research, 20(96), 1-7.
Amiriparian, S., Gerczuk, M., Ottl, S., Cummins, N., Freitag, M., Pugachevskiy, S., . . . Schuller, B. (2017, August). Snore sound classification using imagebased deep spectrum features. In International speech communication association (interspeech) (pp. 3512– 3516).
Bradley, A. P. (1997). The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145-1159. doi: 10.1016/ S0031-3203(96)00142-2
Brown, C., Chauhan, J., Grammenos, A., Han, J., Hasthanasombat, A., Spathis, D., . . . Mascolo, C. (2020). Exploring automatic diagnosis of covid-19 from crowdsourced respiratory sound data. In Proceedings of the 26th acm sigkdd international conference on knowledge discovery and data mining (p. 3474–3484). doi: 10.1145/3394486.3412865
Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Acm sigkdd international conference on knowledge discovery and data mining (p. 785–794). doi: 10.1145/2939672.2939785
Deb, S., Warule, P., Nair, A., Sultan, H., Dash, R., & Krajewski, J. (2022, 9). Detection of common cold from speech signals using deep neural network. Circuits, Systems, and Signal Processing. doi: 10.1007/s00034-022-02189-y
Eyben, F., Wollmer, M., & Schuller, B. (2010, 01). opensmile – the munich versatile and fast open-source audio feature extractor. ACM Multimedia 2010 International Conference, 1459-1462. doi: 10.1145/1873951.1874246
FPT. (1999). Fpt softwave company limited. Retrieved from https://www.fpt-software.com ([Online; accessed 30-04-2023])
Friedman, J. (2000, 11). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29. doi: 10.1214/aos/1013203451
Hoang, T., Pham, L., Ngo, D., & Nguyen, H. D. (2022). A cough-based deep learning framework for detecting covid-19. In The 44th ieee engineering in medicine and biology society (embc) (p. 3422-3425). doi: 10.1109/ EMBC48229.2022.9871179
Islam, R., Abdel-Raheem, E., & Tarique, M. (2022). A novel pathological voice identification technique through simulated cochlear implant processing systems. Applied Sciences, 12(5). doi: 10.3390/app12052398
Kreiman, J., Gerratt, B. R., & Precoda, K. (1990). Listener experience and perception of voice quality. Journal of Speech, Language, and Hearing Research, 33(1), 103– 115.
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. In Eighth ieee international conference on data mining (p. 413-422). doi: 10.1109/ICDM.2008.17
McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., & Nieto, O. (2015). librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference (Vol. 8, pp. 18–25).
Mo, A., Gui, E., & Fletcher, R. R. (2022). Use of voluntary cough sounds and deep learning for pulmonary disease screening in low-resource areas. In Ieee global humanitarian technology conference (ghtc) (p. 242-249). doi: 10.1109/GHTC55712.2022.9911027
NYU Breathing Sounds for COVID-19. (2020). https:// breatheforscience.com/. ([Online; accessed 30-March-2023])
Orlandic, L., Teijeiro, T., & Atienza, D. (2021). The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms. Springer Science and Business Media LLC, 8(1). doi: 10.1038/s41597-021-00937-4
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Peplinski, J., Shor, J., Joglekar, S., Garrison, J., & Patel, S. (2021). FRILL: A Non-Semantic Speech Embedding for Mobile Devices. In International speech communication association (interspeech) (pp. 1204–1208). doi: 10.21437/Interspeech.2021-2070
Pham, L., Ngo, D., Tran, K., Hoang, T., Schindler, A., & McLoughlin, I. (2022). An Ensemble of Deep Learning Frameworks for Predicting Respiratory Anomalies. In Ieee engineering in medicine & biology society (embc) (p. 4595-4598). doi: 10.1109/EMBC48229 .2022.9871440
Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D. (2009). Dataset shift in machine learning. The MIT Press.
Rabanser, S., Gunnemann, S., & Lipton, Z. C. (2019). Failing loudly: An empirical study of methods for detecting dataset shift. Curran Associates Inc.
Rocha, B. M., Filos, D., Mendes, L., Serbes, G., Ulukaya, S., Kahya, Y. P., . . . de Carvalho, P. (2019, mar). An open access database for the evaluation of respiratory sound classification algorithms. Physiological Measurement, 40(3), 035001. doi: 10.1088/1361-6579/ab03ea
Sakkatos, P., Barney, A., Bruton, A., Haitchi, H. M., Kurukulaaratchy, R. J., & Thackray, D. (2019). Quantified breathing patterns can be used as a physiological marker to monitor asthma. European Respiratory Journal, 54(suppl 63). doi: 10.1183/13993003.congress -2019.PA5038
Sasaki, Y. (2007, 01). The truth of the f-measure. Teach Tutor Mater.
Schmitt, M., & Schuller, B. (2017). openxbow – introducing the passau open-source crossmodal bag-ofwords toolkit. Journal of Machine Learning Research, 18(96), 1–5.
Sharma, N., Krishnan, P., Kumar, R., Ramoji, S., Chetupalli, S. R., R., N., . . . Ganapathy, S. (2020). Coswara — A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis. In International speech communication association (interspeech) (pp. 4811–4815). doi: 10.21437/Interspeech.2020-2768
Sharma, N. K., Chetupalli, S. R., Bhattacharya, D., Dutta, D., Mote, P., & Ganapathy, S. (2022). The second dicova challenge: Dataset and performance analysis for diagnosis of covid-19 using acoustics. In Ieee international conference on acoustics, speech and signal processing (icassp) (p. 556-560). doi: 10.1109/ ICASSP43922.2022.9747188
Shor, J., Jansen, A., Maor, R., Lang, O., Tuval, O., de Chaumont Quitry, F., . . . Haviv, Y. (2020). Towards Learning a Universal Non-Semantic Representation of Speech. In International speech communication association (interspeech) (pp. 140–144). doi: 10.21437/ Interspeech.2020-1242
Song, I. (2015). Diagnosis of pneumonia from sounds collected using low cost cell phones. In International joint conference on neural networks (ijcnn) (p. 1-8). doi: 10.1109/IJCNN.2015.7280317
Woolcock institute of medical research vietnam. (1981). Retrieved from https://www.woolcockvietnam .org ([Online; accessed 30-04-2023])
Yang, Y., Yuan, Y., Zhang, G., Hao Wang, Y.-C. C., Liu, Y., Tarolli, C. G., . . . Katabi, D. (2022, august). Artificial intelligence-enabled detection and assessment of parkinson’s disease using nocturnal breathing signals. Nature Medicine. doi: 10.1038/s41591-022-01932-x
Zhao, Y., & Hryniewicki, M. K. (2018). Xgbod: Improving supervised outlier detection with unsupervised representation learning. In International joint conference on neural networks (ijcnn) (p. 1-8). doi: 10.1109/ IJCNN.2018.8489605
Zhao, Y., Nasrullah, Z., & Li, Z. (2019). Pyod: A python toolbox for scalable outlier detection. Journal of Machine Learning Research, 20(96), 1-7.
Section
Regular Session Papers
This work is licensed under a Creative Commons Attribution 3.0 Unported License.