ROC-based Business Case Analysis for Predictive Maintenance – Applications in Aircraft Engine Monitoring

Abstract

Predictive maintenance approaches leveraging integrated knowledge, fleet-wide data and machine-learning techniques allow for earlier warnings on impeding failures and for higher accuracy in remaining useful life predictions compared with traditional prognostics. However, in case relative to correctly predicted maintenance needs, missed detections or false alarms occur too often, follow-up costs, e.g. due to cascading effects or unnecessary inspection effort, can outweigh the advantages. Here, we show that a general cost-benefit analysis based on the Receiver Operating Characteristics (ROC) curve of a failure prediction algorithm allows deducing application-specific requirements on the failure prediction quality for achieving a net benefit. Moreover, various prediction algorithms can be compared and optimized regarding cost-efficiency. The value of the approach is demonstrated by an application example in aircraft engine maintenance showing that for reducing unscheduled engine removals by (more) accurate prediction of turbine blade failures, maximal cost-saving potentials of up to 16 Mio $ emerge per mature-run, widebody engine and per Mean-Time Between Removals (MTBRs). Here, realistic, literature-based assumptions on various costs, failure probability and algorithm performance were incorporated and varied within sensible limits. As a further key result, the value of data for predictive maintenance purposes is impressively demonstrated. Compared with the net benefit achievable by failure prediction of a pure physics-based damage accumulation model and by turbine operating data, an up to 6 Mio. $ higher cost-saving potential per MTBRs was shown to emerge from a literature-based hybrid approach fusing physics and further sources of data, e.g. on manufacturing, geography and environment as well as customer and inspection information by means of machine learning techniques. Generalized applications of the presented cost-benefit analysis approach e.g. to optimize costs associated with engine workscope planning or other system maintenance are discussed.