Analysis Quality Index - What confidence do you have in your risk analysis?



Leila Salhi Jacek Stecki Chris Stecki


OEMs and operators of complex mission/safety critical systems are faced with the requirement to mitigate design and performance risks and their economic consequences. A key issue for any engineering organization is the integrity of the analysis that is used to support significant commercial decisions. Analysis outputs used to establish or validate performance criteria should have an appropriately high level of confidence associated with them when entering into significant financial contracts. While risk assessment methods and techniques for analysis are well defined and understood and are captured in various international military and commercial standards, the issue of analysis quality has traditionally been neglected and is not adequately covered in most commercially available engineering analysis tools. The quality of data inputs determines the quality of analysis outputs. A key factor is the source of the parameters used in an analysis. For example input data may be sourced from operational data, or may be based on the engineering judgement of an individual or a third party organization. This paper outlines an approach to analysis quality assessment in a model based engineering environment, focusing on the sources of data and ancillary information to generate an Analysis Quality Index (AQI) for the analysis. The AQI is generated as a dashboard reporting function for the engineering model that is used to provide a confidence rating on the analysis outputs. Analysis Quality Index capability was incorporated into Maintenance Aware Design environment (MADe) software, an integrated tool-set that combines engineering risk analysis capabilities to support systems engineering, design and through-life support.

How to Cite

Salhi, L. ., Stecki, J. ., & Stecki, C. . (2014). Analysis Quality Index - What confidence do you have in your risk analysis?. Annual Conference of the PHM Society, 6(1).
Abstract 20 | PDF Downloads 13



MADe software, Data, Quality, Index, Analysis

International Standards Organization (ISO) (2004). ISO 31000:2009: Risk Management - Principles and Guidelines. Genève, Switzerland: International Standards Organization

Bednarz, S. and Marriott, D. (1988). Efficient Analysis for FMEA. Proceedings of the 1988 IEEE Annual Reliability and Maintainability Symposium (416-421), January 26-28, Los Angeles, CA, USA.

Bell D., Cox L., Jackson S. & Schaefer, P. (1992). Using Causal Reasoning for Automated Failure Modes and Effects Analysis (FMEA). Proceedings of the I992 Reliability and Maintainability Symposium (343 – 353), January 21-23, Las Vegas, NV, USA.

Carlson. C. S. (2012). Effective FMEAs. Hoboken: John Wily & Sons, Inc.

Evans J. (1992). Editorial - CBIA. IEEE Transactions on Reliability, vol. 41, 1. March 1992.

Evans J. (1992). Editorial - The Demise of R&M. IEEE Transactions on Reliability, vol. 41, 1, . March 1992.

Gilchrist, W. (1993). Modelling failure modes and effects analysis. International Journal of Quality and Reliability Management, 10(5), 16-23.

Hunt J.E., Pugh D.R. & Price C.P. (1995). Failure Mode Effects Analysis: A Practical Application of Functional Modeling. Applied Artificial intelligence, 9( 1), 33-44.

Kara-Zaitri C., Keller A., Barody I. & Fleming, P. (1991), An Improved FMEA Methodology. Proceedings of the 1991 IEEE Annual Reliability and Maintainability Symposium (248-252), January 29-31, Orlando, FL, USA.

McKinney B. (1991). FMECA, The Right Way.Proceedings of the 1991 IEEE Annual Reliability and Maintainability Symposium (253-259), January 29-31, Orlando, FL, USA.

Ormsby A., Hunt J. & Lee M. (1991), Towards an Automated FMEA Assistant. In Rzevski G. & Adey R. (Eds), Applications of Artificial Intelligence in Engineering VI (739-752), Southampton, Boston: Computational Mechanics Publications.

Ross, K. & Main, B.W. (2001). Risk Assessment and Product Liability. Product Liability Committee, 34-38.

Rudov-Clark, S.D & Stecki, J. (2009). The language of FMEA: on the effective use and reuse of FMEA data. Sixth DSTO International Conference on Health & Usage Monitoring, March 9-12, Melbourne, Australia.

Rudov-Clark S., Stecki J. & Stecki C. (2011). Application of advanced failure analysis results for reliability and availability estimations. AERO '11 Proceedings of the 2011 IEEE Aerospace Conference (pp 5), March 5-12, Big Sky, USA.

Wirth, R., Berthold, B., Kramer, A. & Peter, G. (1996). Knowledge-based Support Analysis for the Analysis of Failure Modes and Effects. Engineering Applications of Artificial Intelligence, 9(3), 219-229.
Technical Papers