Benchmarking the Vehicle Integrated Prognostic Reasoner



Xenofon Koutsoukos Gautam Biswas Dinkar A. Mylaraswamy George D. Hadden Daniel Mack Doug Hamilton


This paper outlines a benchmarking approach for evaluating the diagnostic and prognostic capabilities of the Vehicle Integrated Prognostic Reasoner (VIPR), a vehicle-level reasoner and an architecture which aims to detect, diagnose, and predict adverse events during the flight of an aircraft. A number of diagnostic and prognostic metrics exist, but these standards are defined for well-circumscribed algorithms that apply to small subsystems. For layered reasoners, such as VIPR, the overall performance cannot be evaluated by metrics solely directed toward timely detection and accuracy of estimation of the faults in individual components. Among other factors, the overall vehicle reasoner performance is governed by the effectiveness of the communication schemes between the different monitors and reasoners in the architecture, and the ability to propagate and fuse relevant information to make accurate, consistent, and timely predictions at different levels of the reasoner hierarchy. To address these issues, we outline an extended set of diagnostic and prognostics metrics that can be used to evaluate the performance of layered architecture, and we discuss a software architecture*as well as an evaluation plan for benchmarking VIPR.

How to Cite

Koutsoukos, X. ., Biswas, G. ., A. Mylaraswamy, D., Hadden, G. D., Mack, D. ., & Hamilton, D. . (2010). Benchmarking the Vehicle Integrated Prognostic Reasoner. Annual Conference of the PHM Society, 2(1).
Abstract 5 | PDF Downloads 2



diagnostic and prognostic metrics

Byington, C.S., Watson, M., Kalgren, P., & Safa- Bakhsh, R. (2003). Metrics Evaluation and Tool Development for Health and Usage Monitoring System Technology, Third International Conference on Health and Usage Monitoring - HUMS2003, Melbourne, Australia, G.F Forsyth (Ed.), pp. 27-35.
Kurtoglu, T., Mengshoel, O, & Poll, S. (2008). A Framework for Systematic Benchmarking of Monitoring and Diagnostic Systems, International Conference on Prognostics and Health Management Conference, Denver, CO.
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008). Metrics for Evaluating Performance of Prognostic Techniques, in International Conference on Prognostics and Health Management (PHM08), Denver CO.
Leao, B.P., Yoneyama, T., Rocha, G.C., & Fitzgibbon, K.T. (2008). Prognostics performance metrics and their relation to requirements, design, verification and cost-benefit. Proceedings of the International Conference on Prognostics and Health Management (PHM08), Denver, CO.
Kurtoglu, T., Narasimhan, S., Poll, S., Garcia, D., Kuhn, L., de Kleer, J., van Gemund, A., & Feldman, A. (2010). Towards a Framework for Evaluating and Comparing Diagnosis Algorithms. Proceedings of the 20th International Workshop on Principles of Diagnosis, Stockholm, Sweden.
Christensen, M. (2010). Boeing 787 Central Maintenance Computing Function Summary, Technical Report, Honeywell Labs.
NASA (2009). Integrated Vehicle Health Management Technical Plan, Version 2.03, National Aeronautics and Space Administration.
Button, R.M. & Chicatelli, A. (2005). Electrical Power System Health Management, Proc. 1st International Forum on Integrated System Health Engineering and Management in Aerospace, Napa, CA.
Biswas, G., Kapadia, R., & Xu, X.W. (1997). Combined Qualitative–Quantitative Steady-State Diagnosis of Continuous-Valued Systems, IEEE Transactions on Systems, Man, and Cybernetics— Part A: Systems and Humans, 27(2), pp. 167-185.
Poster Presentations

Most read articles by the same author(s)

1 2 > >>