Operational Metrics to Assess Performances of a Prognosis Function. Application to Lubricant of a Turbofan Engine Over- Consumption Prognosis
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
In the aeronautical field, one of the major concerns is the availability of systems. To ensure availability, Prognosis and Health Management algorithms are developed. The aim of these algorithms is twofold. The first one is to detect and locate degradation premise of “no go” condition occurrence. The second one is to predict the health state of the system at a given time horizon. Before introducing PHM algorithms in operation, it is necessary to assess their performances. This is accomplished thank to a “maturation” phase. This phase consists in defining the performance metrics from an operational relevance point of view, in estimating this performance indicator and finally in proposing improvements to meet the airline companies requirements. We consider that the maturation of the detection function has already been completed and that we are interested in the maturation of the prognosis function. This paper deals with the performance assessment of a prognosis function using two operational metrics. A performance estimation procedure is developed. It is applied to the prognosis of turbofan engine lubricant over-consumption. The considered prognosis function is the probability to cross “no go” condition threshold at a given time horizon. This prediction is made thanks to an indicator of the health state of the system. Then it is compared with a threshold in order to trigger an alarm and give rise to a removal if necessary. Within this framework, we have defined two operational metrics for assessing the performance of this prognosis function. These metrics are the “ratio of justified removals” (P(Alarm|Crossing)) and the “ratio of not justified removals” (P(No-crossing|Alarm)). These metrics require the availability of observed lubricant over-consumption to compare the prediction results to the observed values. In the absence of lubricant over-consumption values in operation, a way is to simulate values.
This communication describes the procedure to estimate the performance of the prognosis function and presents the obtained results. The performances estimations trigger improvements. It appears that we have to enhance the precision of the considered health indicator before continuing to assess the performance of the considered prognosis function.
How to Cite
##plugins.themes.bootstrap3.article.details##
maturation, performance evaluation, PHM, failure prognosis, Turbofan engine, Engine Health Monitoring, Prognostic Evaluation
Bröcker J. et Smith L.A. (2007). Increasing the reliability of reliability diagrams. Wea. Forecasting, 22, pp 651-661.
Candille G. et Talagrand O. (2005). Evaluation of probabilistic prediction systems for a scalar variable. Royal Meteorological Society, pp 1-20.
Demaison F., Flandrois X., Massé J.R., Massot G., Hmad O. & Ricordeau J. (2011). Method and system for monitoring the level of oil contained in a tank of an aircraft engine. Patent # WO/2011/131892.
Dragomir O.E., Gouriveau R. et Zerhouni, N. (2008). Adaptive neuro-fuzzy inference system for midterm prognostic error stabilization. International Journal of Computers, Communications and Control, pp 271-276.
Ebert B. Forecast Verification - Issues, Methods and FAQ. Last access 2013, url: http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html.
Hmad O., Grall-Maes E., Beauseroy P., Masse J.R. & Mathevet A. (2012). A Maturation Procedure for Prognosis and Health Monitoring Algorithms. PSAM11 & ESREL12 conference, June 25-29, Helsinki, Finland.
Jardine A.K.S., Lin D. et Banjevic D. (2006). A review on machinery diagnostics and prognostics implementing condition based maintenance. Mechanical Systems and Signal Processing, 20, pp 1483-1510.
Letot C. et Dehombreux P. (2009). Modèles de dégradation pour l’estimation de la fiabilité et de la durée de vie résiduelle : applications à la fissuration. Actes du 4ème congrès PENTOM, Autrans, France.
Murphy A.H. (1973). A new vector partition of the probability score. J. Appl. Meteor., 12, pp 595-600.
Nikulin M.S., Limnios N., Balakrishnan N., Kahle W. et Huber-Carol C. (2010). Advances in Degradation Modeling Applications to Reliability, Survival Analysis, and Finance. Birkhauser, Boston.
Roussignol M. (2009). Gamma stochastic process and application to maintenance. Cours de l’université Paris Est – Marne la vallée.
Saxena A., Celaya J., Balaban E., Goebel K., Saha B., Saha S. et Schwabacher M. (2008). Metrics for Evaluating Performance of Prognostics Techniques. 1st International Conference on Prognostics and Health Management, Denver, USA.
Saxena A., Celaya J., Goebel K., Saha B. et Saha S. (2009). Evaluating Algorithm Performance Metrics Tailored for Prognostics. IEEE Aerospace Conference, Big Sky Montana, pp 1-13.
Si X.S, Wang W., Hu C.H., Zhou D.H. (2011). Remaining useful life estimation – A review on the statistical data driven approaches. European Journal of Operational Research, Volume 213, Issue 1, pp 1-14. ISSN 0377-2217.
Sikorska J.Z., Hodkiewicz M., Ma L.. (2011). Prognostic modelling options for remaining useful life estimation by industry. Mechanical Systems and Signal Processing, Volume 25, Issue 5, pp 1803-1836. ISSN 0888-3270.
Vachtsevanos G., Lewis F.L., Roemer M., Hess A. et Wu B. (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems. First edition, John Wiley & Sons, Hoboken, New Jersey.
Van Noortwijk J.M. (2009). A survey of the application of gamma processes in maintenance. Reliability Engineering and System Safety, 94(1), pp 2–21.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.