CBM/PBM Operational Performance: Concepts & Applicable Formulas For Aligning Models Performance Metrics With Operational Ones At Failure Mode And Equipment Levels

Abstract

Condition-Based Maintenance (CBM) and Predictive-Based Maintenance (PBM) have emerged as pivotal strategies in the
aircraft industry, offering the potential to revolutionize maintenance operations. These approaches aim to optimize maintenance schedules, reduce operational costs, increase aircraft and assets availability, and enhance safety by leveraging Health Indicators (HIs). These indicators enable the monitoring of the aircraft health throughout its lifecycle, supporting the prediction of potential failures and proactive maintenance interventions. Consequently, a critical aspect of realizing the full benefits of CBM/PBM lies in the ability to accurately measure the performance of both predictive models and the operational performance of the equipment itself. However, the widespread and effective implementation of CBM/PBM faces challenges. Difficulties often arise from the inherent complexity of these systems and the need for efficient collaboration among stakeholders with diverse backgrounds and business objectives. In particular, the concept of "performance" can be interpreted differently by various stakeholders, leading to misunderstandings and hindering effective decision-making. 
This paper addresses the limitations of relying solely on conventional operational metrics, such as the MTBF (Mean Time Between Failures), within a CBM/PBM context. It recognizes the necessity to move beyond traditional concepts and metrics associated with reactive maintenance and embrace a more holistic view. To bridge the gap between different perspectives, this work proposes an adapted framework that facilitates the reconciliation of diverse viewpoints. The proposed approach encompasses a detailed understanding of internal failure modes and the establishment of appropriate correspondence laws at equipment level. Ultimately, it seeks to foster stronger communication and understanding within the broader ecosystem of stakeholders, including suppliers, Maintenance, Repair and Overhaul (MRO) providers, and end-users.
To achieve this, this paper introduces a set of new operational metrics as a paradigm shift, specifically designed for CBM/PBM environments. The MTBPR (Mean Time Between Predictive-based preventive Removals), which quantifies the time between preventive removals based on predictions; the NDF (No Degradation Found) rate, which measures the rate of early removals where no degradation was detected; the MLR (Mean Lifetime Reduction), which accounts for the reduction in equipment lifetime due to early removals. Importantly, the paper establishes clear analogies between these new metrics and the legacy metrics used in unscheduled maintenance strategies, namely the MTBUR (Mean Time Between Unscheduled Removals) and the NFF (No Fault Found) rate.
Furthermore, this paper delves into the crucial relationship between the performance of predictive models and operational performance. It demonstrates how typical data analytics metrics, such as Recall and Precision, can be effectively linked to classic equipment-level reliability metrics. By establishing these formal connections, this paper provides a generalized and robust approach to support the definition of performance objectives for predictive monitoring, and the way to properly monitor in-service performance (feedback loop). This approach fosters alignment and collaboration among all stakeholders involved in the CBM/PBM domain, ultimately leading to more effective and harmonized maintenance strategies and improved operational outcomes.