Prognostic Algorithm Verification



N. Scott Clements David S. Bodden


A rigorous methodology is presented for both specification and verification of prognostic algorithm performance. The prognostic algorithm specification statement takes the form, “The prognostic algorithm shall provide a minimum of <TTM> hours time-to-maintenance such that between <Lower>% and <Upper>% of failures of component ABC will be avoided with <Confidence>% confidence.” The methodology is developed first for a single failure mode case and then extended to the multiple failure mode case. The case of non-prognosable failure modes is also considered. Finally, implications of this approach are presented, including pre- tabulation of confidence bounds, estimation of the minimum amount of data required to reach a given verification confidence, and a method for using a minimum confidence growth curve to account for initial low confidence in a prognostic algorithm.

How to Cite

Clements , N. S. ., & S. Bodden, D. . (2013). Prognostic Algorithm Verification. Annual Conference of the PHM Society, 5(1).
Abstract 21 | PDF Downloads 14



prognostics, Requirements, Verification, Confidence

Azzam, H., Cook, J., & Driver, S. (2004), FUMSTM technologies for verifiable affordable prognostics health management (PHM), IEEE Aerospace Conference Proceedings (vol. 6, pp. 3764-3781), March 6-13.

Feldman, A., Kurtoglu, T., Narasimhan, S., Poll, S., Garcia, D., de Kleer, J., Kuhn, L. & van Gemund, A. (2010), Empirical Evaluation of Diagnostic Algorithm Performance Using a Generic Framework, International Journal of the PHM Society (vol. 1).

Halligan, G. R., Thumati, B. T., & S. Jagannathan (2011), Artificial immune system-based diagnostics and prognostics scheme and its experimental verification, IEEE International Conference on Control Applications (CCA) (pp. 958-963), 28-30 Sept. 2011.

Hao, L., Jinsong, Y., Ping, Z., & Xingshan, L. (2011), Review on Verification and Validation technology in integrated health management system, 10th International Conference on Electronic Measurement & Instruments (ICEMI) (vol. 2, pp. 125-129), August 16- 19.

Kacprzynski, G. J., Liberson, A., Palladino, A., Roemer, M. J., Hess, A. J., & Begin, M. (2004), Metrics and development tools for prognostic algorithms, IEEE Aerospace Conference Proceedings (vol. 6, pp. 3809- 3815), March 6-13.

Leao B. P., Yoneyama, T., Rocha, G. C., & Fitzgibbon, K. T. (2008), Prognostics Performance Metrics and their Relation to Requirements, Design, Verification and Cost-Benefit, IEEE International Conference on Prognostics and Health Management.

Line, J. K., & Clements, N. S. (2006), Prognostics usefulness criteria, IEEE Aerospace Conference Proceedings, March.

Luna, J. J. (2010), Towards Defining and Allocating PHM Requirements for Military Systems, Annual Conference of the Prognostics and Health Management Society.

Lybeck, N., Marble, S., & Morton, B. (2007), Validating Prognostic Algorithms: A Case Study Using Comprehensive Bearing Fault Data, IEEE Aerospace
Conference (pp. 1-9), March 3-10.

Massam, S., & McQuillan, S. (2002), Verification of PHM capabilities: A joint customer/industrial perspective, IEEE Aerospace Conference Proceedings (vol. 6, pp. 2799-2813).

Mikat, H., Siddiolo, A. M., & Buderath, M. (2012), Virtual Framework for Validation and Verification of System Design Requirements to enable Condition Based Maintenance, First European Conference of the Prognostics and Health Management Society.

Ping, X., Wang, Z., & Vue, L. (2010), Prognostics and Health Management (PHM) System requirements and validation, Prognostics and Health Management Conference (pp. 1-4), January 12-14.

Roemer, M. J., Dzakowic, J., Orsagh, R. F., Byington, C. S., & Vachtsevanos, G. (2005), Validation and verification of prognostic and health management technologies, IEEE Aerospace Conference (pp. 3941-3947), March 5- 12.

Saxena A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008), Metrics for Evaluating Performance of Prognostic Techniques, IEEE International Conference on Prognostics and Health Management, October 6-9.

Saxena, A., Celaya, J., Saha, B., Saha, S., & Goebel, K. (2010), Metrics for Offline Evaluation of Prognostic Performance, International Journal of the PHM Society (vol. 1).

Tang, L., Orchard, M. E., Goebel, K., & Vachtsevanos, G. (2011), Novel metrics and methodologies for the verification and validation of prognostic algorithms, IEEE Aerospace Conference (pp. 1-8), March 5-12.

Usynin, A., Hines, J. W., & Urmanov, A. (2007), Formulation of Prognostics Requirements, IEEE Aerospace Conference (pp. 1-8), March 3-10.

Wheeler, K. R., Kurtoglu, T., & Poll, S. D. (2010), A Survey of Health Management User Objectives Related to Diagnostic and Prognostic Metrics, International Journal of the PHM Society (vol. 1).

Zhang, B., Tang, L., DeCastro, J., & Goebel, K. (2010), A verification methodology for prognostic algorithms, IEEE AUTOTESTCON (pp. 1-8), September 13-16.

Zhou, Y., Bo, J., Jie, Z., & Mingwei, G. (2013), Performance Metrics Assessment Method on Aircraft Prognostics and Health Management, Fifth International Conference on Measuring Technology and Mechatronics Automation (ICMTMA) (pp. 799-802), January 16-17.
Technical Papers

Most read articles by the same author(s)