SYSAI for System Health Management - a Statistical Framework for the Analysis of Diagnosis Systems
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
On-board failure diagnosis and health management systems (HMS) are crucial for the operation of complex autonomous aerospace systems. False alarms (false positives, FPs) or false negatives (FNs) can lead to lower system performance or even loss of mission or the autonomous vehicle. Therefore, a careful verification and validation (V&V) is important. Due to the high dimensionality of the system’s state space, however, exhaustive testing of the HMS is usually not possible.
In this paper, we present how our SYSAI (System Analysis for Systems with AI components) framework can support intelligent analysis and testing of HMS on the system level. SYSAI’s capabilities to efficiently explore high-dimensional state and parameter spaces and to identify diagnosability regions and their boundaries, makes a comprehensive analysis of the diagnosis system possible and can provide feedback to the designer. We will illustrate our approach using the ADAPT (Advanced Diagnostics and Prognostics Testbed) redundant power storage and distribution system.
How to Cite
##plugins.themes.bootstrap3.article.details##
statistical analysis, verification and validation, runtime assurance
Barua, A., & Khorasani, K. (2012). Verification and validation of hierarchical fault diagnosis in satellites formation flight. IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews). doi: 10.1109/tsmcc.2012.2187188
Batteux, M., Dague, P., Rapin, N., & Fiani, P. (2011). Diagnosability study of technological systems. doi: 10.1007/978-3-642-21822-4\ 20
Chen, Y.-Y., & Wu, G.-W. (2007). Fault-tolerant verification platform for systems modeled at high level of abstraction. doi: 10.1109/systems.2007.374697
Cimatti, A., Pecheur, C., & Cavada, R. (2003). Formal verification of diagnosability via symbolic model checking. In Proc. IJCAI (p. 363-369).
Clarke, E. M., Grumberg, O., & Peled, D. A. (2000). Model checking. Cambridge, MA, USA: MIT Press.
Cohn, D. A. (1996). Neural network exploration using optimal experimental design. Advances in Neural Information Processing Systems, 6(9), 679--686.
Farsoni, S., & Simani, S. (2021). Validation of fault diagnosis techniques based on artificial intelligence tools for a wind turbine benchmark. In 2021 5th international Conference on Control and Fault-tolerant Systems (SYSTOL) (p. 157-162). doi: 10.1109/SysTol52990.2021 .9595291
Gertler, J. (2021). Fault detection and diagnosis. In Encyclopedia of Systems and Control (pp. 764--769). Springer.
Gramacy, R., & Polson, N. (2011). Particle learning of Gaussian process models for sequential design and optimization. Journal of Computational and Graphical Statistics, 20(1), 467--478.
He, Y. (2012). Variable-length functional output prediction and boundary detection for an adaptive flight control simulator (Doctoral dissertation). University of California at Santa Cruz.
He, Y. (2015). Online detection and modeling of safety boundaries for aerospace applications using active learning and bayesian statistics. In 2015 International joint Conference on Neural Networks, IJCNN 2015 (pp. 1--8). IEEE. Retrieved from http://ieeexplore.ieee.org/xpl/ mostRecentIssue.jsp?punumber=7256526 doi: 10.1109/IJCNN.2015.7280595
He, Y., & Schumann, J. (2020). A framework for the analysis of deep neural networks in aerospace applications using bayesian statistics. In Proc. IJCNN, WCCI.
He, Y., Yu, H., Brat, G., & Davies, M. (2021). Statistical learning framework for safety and failure analysis of a DNN-based autonomous aircraft system. In Proc. International Conference on Machine Learning Applications (ICMLA), IEEE.
He, Y., Yu, H., Brat, G., & Davies, M. (2022). System and safety analysis for autonomous center line tracking with SYSAI. In SciTech 2022,
Jones, D., Schonlau, M., & Welch, W. J. (1998). Efficient global optimization of expensive black box functions. Journal of Global Optimization, 13, 455--492.
Knox, W., & Mengshoel, O. (2009). Diagnosis and reconfiguration using bayesian networks: An electrical power system case study. In Proc. IJCAI.
Kurtoglu, T., Jensen, D., & Poll, S. (2009). Systematic benchmarking of diagnostic technologies for an electrical power system. InIEEE Aerospace Conference (p. 1 - 9). doi: 10.1109/AERO.2009.4839623
Lei, Y., Yang, B., Jiang, X., Jia, F., Li, N., & Nandi, A. K. (2020). Applications of machine learning to machine fault diagnosis: A review and roadmap. Mechanical Systems and Signal Processing, 138, 106587. doi: https://doi.org/10.1016/j.ymssp.2019.106587
Lincoln, P., & Rushby, J. (1993). The formal verification of an algorithm for interactive consistency under a hybrid fault model. doi: 10.1007/3-540-56922-7\ 24
MacKay, D. J. C. (1992). Information--based objective functions for active data selection. Neural Computation, 4(4), 589--603.
Mahadevan, N., Lowry, M., Schumann, J., & Karsai, G. (2016). Dver: A tool chain for cross-validation and perfection of discrete model-based diagnostic systems. In 2016 IEEE aerospace conference (p. 1-15). doi: 10.1109/AERO.2016.7500913
McComas, D. (2012). NASA/GSFC’s Flight Software Core Flight System. In Flight software workshop.
Moradi, M., Van Acker, B., Vanherpen, K., & Denil, J. (2019). Model-implemented hybrid fault injection for simulink (tool demonstrations). In (p. 71-90). doi: 10.1007/ 978-3-030-23703-5 4
Poll, S., Patterson-Hine, A., Camisa, J., Garcia, D., Hall, D., Lee, C., . . . Koutsoukos, X. (2007). Advanced diagnostics and prognostics testbed. In 18th international workshop on principles of diagnosis.
Qualtech. TEAMS designer. Retrieved from http://www.teamqsi.com/products/ teams-designer/
Ranjan, P., Bingham, D., & Michailidis, G. (2008). Sequential experiment design for contour estimation from complex computer codes. Technometrics, 50(4), 527--541.
Schumann, J., Gomez-Gonzalez, V., Mahadevan, N., Lowry, M., Robinson, P., & Karsai, G. (2014). A tool chain for the v&v of nasa cryogenic fuel loading health management. In Annual conference of the phm society, 6(1).
Su, J., & Chen, W.-H. (2019). Model-based fault diagnosis system verification using reachability analysis. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(4), 742-751. doi: 10.1109/TSMC.2017.2710132
Taddy, M. A., Gramacy, R. B., & Polson, N. G. (2011). Dynamic trees for learning and design. Journal of the American Statistical Association, 106(493), 109-123. VirtualADAPT. Retrieved from https://github.com/ nasa/VirtualADAPT
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.