This paper outlines a benchmarking approach for evaluating the diagnostic and prognostic capabilities of the Vehicle Integrated Prognostic Reasoner (VIPR), a vehicle-level reasoner and an architecture which aims to detect, diagnose, and predict adverse events during the flight of an aircraft. A number of diagnostic and prognostic metrics exist, but these standards are defined for well-circumscribed algorithms that apply to small subsystems. For layered reasoners, such as VIPR, the overall performance cannot be evaluated by metrics solely directed toward timely detection and accuracy of estimation of the faults in individual components. Among other factors, the overall vehicle reasoner performance is governed by the effectiveness of the communication schemes between the different monitors and reasoners in the architecture, and the ability to propagate and fuse relevant information to make accurate, consistent, and timely predictions at different levels of the reasoner hierarchy. To address these issues, we outline an extended set of diagnostic and prognostics metrics that can be used to evaluate the performance of layered architecture, and we discuss a software architecture*as well as an evaluation plan for benchmarking VIPR.
How to Cite
diagnostic and prognostic metrics
Kurtoglu, T., Mengshoel, O, & Poll, S. (2008). A Framework for Systematic Benchmarking of Monitoring and Diagnostic Systems, International Conference on Prognostics and Health Management Conference, Denver, CO.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for Evaluating Performance of Prognostic Techniques, in International Conference on Prognostics and Health Management (PHM08), Denver CO.
Leao, B.P., Yoneyama, T., Rocha, G.C., & Fitzgibbon, K.T. (2008). Prognostics performance metrics and their relation to requirements, design, verification and cost-benefit. Proceedings of the International Conference on Prognostics and Health Management (PHM08), Denver, CO.
Kurtoglu, T., Narasimhan, S., Poll, S., Garcia, D., Kuhn, L., de Kleer, J., van Gemund, A., & Feldman, A. (2010). Towards a Framework for Evaluating and Comparing Diagnosis Algorithms. Proceedings of the 20th International Workshop on Principles of Diagnosis, Stockholm, Sweden.
Christensen, M. (2010). Boeing 787 Central Maintenance Computing Function Summary, Technical Report, Honeywell Labs.
NASA (2009). Integrated Vehicle Health Management Technical Plan, Version 2.03, National Aeronautics and Space Administration.
Button, R.M. & Chicatelli, A. (2005). Electrical Power System Health Management, Proc. 1st International Forum on Integrated System Health Engineering and Management in Aerospace, Napa, CA.
Biswas, G., Kapadia, R., & Xu, X.W. (1997). Combined Qualitative–Quantitative Steady-State Diagnosis of Continuous-Valued Systems, IEEE Transactions on Systems, Man, and Cybernetics— Part A: Systems and Humans, 27(2), pp. 167-185.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.