Towards Explainable Anomaly Detection in Safety-critical Systems Employing FRAM and SpecTRM in International Space Station Telemetry
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Oct 8, 2024
Shota Iino
Hideki Nomoto
Takashi Fukui
Yohei Yagisawa
Sayaka Ishizawa
Takayuki Hirose
Yasutaka Michiura
Abstract
Ensuring the reliability and safety of space missions necessitates advanced anomaly detection systems capable of not only identifying deviations but also providing clear, understandable insights into their causes. This paper introduces a novel methodology for the detection of systemic anomalies in the telemetry data of the International Space Station (ISS), leveraging the synergistic application of the Functional Resonance Analysis Method (FRAM) and the Specification Tools and Requirement Methodology-Requirement Language (SpecTRM-RL). Integrated with machine learning-based normal behavior prediction model, this approach significantly enhances the explanatory of anomaly detection mechanisms. The methodology is verified and validated through its application to the thermal control system within the ISS's Japanese Experimental Module (JEM), illustrating its capacity to augment diagnostic capabilities and assist flight controllers and specialists in preserving the ISS's functional integrity. The findings underscore the importance of explainability in the machine learning-based anomaly detection of safety-critical systems and suggest a promising avenue for future explorations aimed at bolstering space system health management through improved explainability and operational efficiency.
##plugins.themes.bootstrap3.article.details##
Keywords
Explainable, Interpretable, Anomaly Detection, aerospace
References
Ahn, H., Jung, D., & Choi, H. L. (2020). Deep generative models-based anomaly detection for spacecraft control systems. Sensors, 20(7), 1991.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58.
Fisher, A., Rudin, C., & Dominici, F. (2019). All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously. J. Mach. Learn. Res., 20(177), (p. 1-81).
Fuertes, S., Picart, G., Tourneret, J. Y., Chaari, L., Ferrari, A., & Richard, C. (2016). Improving spacecraft health monitoring with automatic anomaly detection techniques. In 14th International Conference on Space Operations. 16-20 May 2016, Korea.
Hayton, P., Utete, S., King, D., King, S., Anuzis, P., & Tarassenko, L. (2007). Static and dynamic novelty detection methods for jet engine health monitoring. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 365(1851), 493-514.
Hirose, T., & Sawaragi, T. (2020). Extended FRAM model based on cellular automaton to clarify complexity of socio-technical systems and improve their safety. Safety science, 123, 104556.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), (p.1735-1780).
Hollnagel, E. (2012). FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-technical Systems. Crc Press.
Hollnagel, E., Wears, R. L., & Braithwaite, J. (2015). From Safety-I to Safety-II: a white paper. The resilient health care net: the University of Southern Denmark, University of Florida, USA, and Macquarie University, Australia.
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018). Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. August 19–23, 2018, London, United Kingdom. doi:10.1145/3219819.3219845
Iino, S., Nomoto, H., T., Sasaki, M., Ishizawa, Y., Fukui, S, Michiura, Y., Hirose, ... & Wada, M. (2022). Explainable symptom detection in telemetry of ISS with Random Forest and SpecTRM. Proceedings of 2022 IEEE Aerospace Conference. 5-12 March, Big Sky, MO. doi: 10.1109/AERO53065.2022.9843739
Iino, S., Nomoto, Fukui, T., Yohei, Y., Ishizawa, Y., H., Hirose, T., Michiura, Y., ... & Shibayama, H. (2023). Time-series anomaly detection in telemetry of ISS providing the reasons with FRAM and SpecTRM. Proceedings of 2023 IEEE Aerospace Conference. 4-11 March, Big Sky, MO.
doi: 10.1109/AERO55745.2023.10115985
Japan Aerospace Exploration Agency (JAXA). (2013). Japanese Manned Space Technology brought by Development and Operation of International Space Station (ISS), Japanese Experiment Module KIBO. JAXA-SP-12-015, 1 – 227.
Kostovska, A., Petković, M., Stepišnik, T., Lucas, L., Finn, T., Martínez-Heras, J., & Kocev, D. (2021). GalaxAI: Machine learning toolbox for interpretable analysis of spacecraft telemetry data. Proceedings of 2021 IEEE 8th International Conference on Space Mission Challenges for Information Technology (SMC-IT). 26-30 July 2021. Pasadena, CA. doi: 10.1109/SMC-IT51442.2021.00013
Leveson, N. G., Reese, J. D., & Heimdahl, M. P. (2003). SpecTRM: A CAD system for digital automation. Proceedings of Digital Avionics Systems Conference. 31 October 1998 - 07 November 1998. Bellevue, WA. doi: 10.1109/DASC.1998.741474
Leveson, N. (2012). Engineering a Safer World: Systems Thinking Applied to Safety (Engineering Systems). The MIT Press.
Malhotra, P., Vig, L., Shroff, G., & Agarwal, P. (2015). Long short term memory networks for anomaly detection in time series. Proceedings of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. 22-24 April 2015, Bruges Belgium. ISBN 978-287587014-8
Morgan, J.N., Sonquest, J.A. (1963). Problems in the analysis of survey data and a proposal. J. Amer. Statist. Assoc. 58, 415–435.
Patriarca, R., Di Gravio, G., & Costantino, F. (2017). A Monte Carlo evolution of the Functional Resonance Analysis Method (FRAM) to assess performance variability in complex systems. Safety science, 91, 49-60.
Pilastre, B., Boussouf, L., d’Escrivan, S., & Tourneret, J. Y. (2020). Anomaly detection in mixed telemetry data using a sparse representation and dictionary learning. Signal Processing, 168, 107320.
Popov, A., Fink, W., Hess, A., & Tarbell, M. A. (2019). A paradigm shift from telemedicine to autonomous human health and performance for long-duration space missions. International Journal of Prognostics and Health Management, 10, 001.
Pugh, S. (1981, March). Concept selection: a method that works. Proceedings of the International conference on Engineering Design (p. 497-506).
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Preprint.
Ross, D. (1977). Structured Analysis: A Language for Communicating Ideas. IEEE Transactions on Software Engineering, 3(1), Special Issue on Requirements Analysis, 16-34.
Wang, C., Lu, N., Cheng, Y., & Jiang, B. (2019). A telemetry data based diagnostic health monitoring strategy for in-orbit spacecrafts with component degradation. Advances in Mechanical Engineering, 11(4), 1687814019839599.
Wyatt, E., & Nilsen, E. (2000). Flight System Autonomy Needs for Planetary Exploration Missions.
Yairi, T., Takeishi, N., Oda, T., Nakajima, Y., Nishimura, N., & Takata, N. (2017). A data-driven health monitoring method for satellite housekeeping data based on probabilistic clustering and dimensionality reduction. IEEE Transactions on Aerospace and Electronic Systems, 53(3), 1384-1401.
Zeng, Z., Jin, G., Xu, C., Chen, S., & Zhang, L. (2022). Spacecraft Telemetry Anomaly Detection Based on Parametric Causality and Double-Criteria Drift Streaming Peaks over Threshold. Applied Sciences, 12(4), 1803.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58.
Fisher, A., Rudin, C., & Dominici, F. (2019). All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously. J. Mach. Learn. Res., 20(177), (p. 1-81).
Fuertes, S., Picart, G., Tourneret, J. Y., Chaari, L., Ferrari, A., & Richard, C. (2016). Improving spacecraft health monitoring with automatic anomaly detection techniques. In 14th International Conference on Space Operations. 16-20 May 2016, Korea.
Hayton, P., Utete, S., King, D., King, S., Anuzis, P., & Tarassenko, L. (2007). Static and dynamic novelty detection methods for jet engine health monitoring. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 365(1851), 493-514.
Hirose, T., & Sawaragi, T. (2020). Extended FRAM model based on cellular automaton to clarify complexity of socio-technical systems and improve their safety. Safety science, 123, 104556.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), (p.1735-1780).
Hollnagel, E. (2012). FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-technical Systems. Crc Press.
Hollnagel, E., Wears, R. L., & Braithwaite, J. (2015). From Safety-I to Safety-II: a white paper. The resilient health care net: the University of Southern Denmark, University of Florida, USA, and Macquarie University, Australia.
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018). Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. August 19–23, 2018, London, United Kingdom. doi:10.1145/3219819.3219845
Iino, S., Nomoto, H., T., Sasaki, M., Ishizawa, Y., Fukui, S, Michiura, Y., Hirose, ... & Wada, M. (2022). Explainable symptom detection in telemetry of ISS with Random Forest and SpecTRM. Proceedings of 2022 IEEE Aerospace Conference. 5-12 March, Big Sky, MO. doi: 10.1109/AERO53065.2022.9843739
Iino, S., Nomoto, Fukui, T., Yohei, Y., Ishizawa, Y., H., Hirose, T., Michiura, Y., ... & Shibayama, H. (2023). Time-series anomaly detection in telemetry of ISS providing the reasons with FRAM and SpecTRM. Proceedings of 2023 IEEE Aerospace Conference. 4-11 March, Big Sky, MO.
doi: 10.1109/AERO55745.2023.10115985
Japan Aerospace Exploration Agency (JAXA). (2013). Japanese Manned Space Technology brought by Development and Operation of International Space Station (ISS), Japanese Experiment Module KIBO. JAXA-SP-12-015, 1 – 227.
Kostovska, A., Petković, M., Stepišnik, T., Lucas, L., Finn, T., Martínez-Heras, J., & Kocev, D. (2021). GalaxAI: Machine learning toolbox for interpretable analysis of spacecraft telemetry data. Proceedings of 2021 IEEE 8th International Conference on Space Mission Challenges for Information Technology (SMC-IT). 26-30 July 2021. Pasadena, CA. doi: 10.1109/SMC-IT51442.2021.00013
Leveson, N. G., Reese, J. D., & Heimdahl, M. P. (2003). SpecTRM: A CAD system for digital automation. Proceedings of Digital Avionics Systems Conference. 31 October 1998 - 07 November 1998. Bellevue, WA. doi: 10.1109/DASC.1998.741474
Leveson, N. (2012). Engineering a Safer World: Systems Thinking Applied to Safety (Engineering Systems). The MIT Press.
Malhotra, P., Vig, L., Shroff, G., & Agarwal, P. (2015). Long short term memory networks for anomaly detection in time series. Proceedings of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. 22-24 April 2015, Bruges Belgium. ISBN 978-287587014-8
Morgan, J.N., Sonquest, J.A. (1963). Problems in the analysis of survey data and a proposal. J. Amer. Statist. Assoc. 58, 415–435.
Patriarca, R., Di Gravio, G., & Costantino, F. (2017). A Monte Carlo evolution of the Functional Resonance Analysis Method (FRAM) to assess performance variability in complex systems. Safety science, 91, 49-60.
Pilastre, B., Boussouf, L., d’Escrivan, S., & Tourneret, J. Y. (2020). Anomaly detection in mixed telemetry data using a sparse representation and dictionary learning. Signal Processing, 168, 107320.
Popov, A., Fink, W., Hess, A., & Tarbell, M. A. (2019). A paradigm shift from telemedicine to autonomous human health and performance for long-duration space missions. International Journal of Prognostics and Health Management, 10, 001.
Pugh, S. (1981, March). Concept selection: a method that works. Proceedings of the International conference on Engineering Design (p. 497-506).
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Preprint.
Ross, D. (1977). Structured Analysis: A Language for Communicating Ideas. IEEE Transactions on Software Engineering, 3(1), Special Issue on Requirements Analysis, 16-34.
Wang, C., Lu, N., Cheng, Y., & Jiang, B. (2019). A telemetry data based diagnostic health monitoring strategy for in-orbit spacecrafts with component degradation. Advances in Mechanical Engineering, 11(4), 1687814019839599.
Wyatt, E., & Nilsen, E. (2000). Flight System Autonomy Needs for Planetary Exploration Missions.
Yairi, T., Takeishi, N., Oda, T., Nakajima, Y., Nishimura, N., & Takata, N. (2017). A data-driven health monitoring method for satellite housekeeping data based on probabilistic clustering and dimensionality reduction. IEEE Transactions on Aerospace and Electronic Systems, 53(3), 1384-1401.
Zeng, Z., Jin, G., Xu, C., Chen, S., & Zhang, L. (2022). Spacecraft Telemetry Anomaly Detection Based on Parametric Causality and Double-Criteria Drift Streaming Peaks over Threshold. Applied Sciences, 12(4), 1803.
Section
Technical Papers