Threshold Selection for Classification Models in Prognostics

Abstract

In this study, we evaluate the performance of a prognostic classification model for NOX sensors in diesel engines over one month by comparing its predictions against actual outcomes. We then construct a validation dataset to assess the model's performance. By analyzing instances where the model's predictions were incorrect, we determine new threshold values that could potentially reduce errors for each false positive (FP) and false negative (FN). Subsequently, we create a dataset where the threshold varies for each observation and train a regression model with the modified threshold as the target variable. Our findings indicate that incorporating this approach, where the model's performance is iteratively refined using the validation dataset, leads to a reduction in both false positives and false negatives.