Predictive maintenance refers to the set of techniques used to determine the operational condition of a system and estimate the maintenance time frame using predictive models. This helps reduce the operational costs of maintenance and prevent untimely failures which could lead to extended downtimes.

An important aspect of predictive maintenance is failure prediction using historical data. Analysis of sensor and telemetry data can be used to predict the Time-To-Failure (TTF) of a system which allows for the planning of a maintenance schedule. This is particularly useful for aircraft manufacturers and operators.

The aim of this exercise is to describe the usage of statistical learning techniques to historical sensor measurement data to predict the future failures of engines. Machine learning techniques can be used to establish relationships between sensor measurements and historical failures. The goal is to make the following predictions:

1. The Time-To-Failure (TTF) of an engine.
2. Classify whether an engine will fail in a given time period.

The data used for this exercise is a subset of a larger dataset generated by Microsoft and consists of run-to-failure scenarios for a number of aircraft engines. The following data files were provided for building the predictive maintenance models:

1. train_selected.csv - contains the historical sensor measurements and failure data for multiple engines over cycles of operation
2. test_selected_ttf.csv - contains the historical sensor measurements and failure data for multiple engines at a randomly selected cycle of operation. This will be used to quantify the accuracy of the models.

In order to achieve the goals of the exercise supervised learning techniques will be used to establish relationships between sensor measurements and historical failures. The two requirements of the problem will be formulated as follows:

1. Regression models to predict the continuous dependent variable TTF
2. Binary classification models to predict whether an engine will fail in a given time period.

As per our analysis, non-linear models outperformed the linear model for the regression task. Polynomial regression and Random Forest models had similar performance on the test dataset.

The Random Forest model took a significant amount of time to train due to the number of hyperparameters whereas the linear and polynomial regression models were trained in a short time. Both the non-linear models show a certain amount of overfitting. For the random forest model, the model can be tuned further to reduce the overfitting whereas for the polynomial regression model, a lower degree can be used, but it reduces the performance the model. Overfitting can also be reduced by increasing the size of the dataset.

The assumptions regarding the performance improvement by using sensors S1 and S3 along with CYCLE and setting an upper threshold for TTF also hold true for all the models. Using the TTF from when degradation actually started gives the optimal results.

While performing binary classification, the Random Forest model had the largest area under the ROC curve. Logistic regression had a better precision than other models while Naive Bayes model had the best recall.

It can be observed that for higher thresholds, Random Forest has better precision than the other models but lower recall. This can be seen in the precision-recall curves. At the selected thresholds, Naive Bayes model has the highest maintenance capacity. A change in thresholds can increase the maintenance capacity but with observable effects on the performance metrics.

In this document, regression and classification models were used for failure analysis for predictive maintenance. Regression models were used to estimate the time to failure (TTF) and classification models were used to predict if an aircraft will fail in a given period. Multiple models were assessed, and the document covers the quantitative analysis of the results. Possible extensions and improvements to the approach are:

• Smoothing the sensor measurements to reduce the effect of noise and make faults more distinguishable.
• Modelling the regression problem as time series analysis and adding distributed lags to the data (Zhang et al., 2018).
• Usage of ARIMA models for forecasting failure of the aircrafts (Yang et al., 2017).
• Employing deep learning techniques such as CNN and LSTM (Hermawan et al., 2020) along with self- organizing maps. (Chen et al., 2007).