The airline industry is undeniably massive with an annual revenue exceeding $800B and 6M travelers per day. A team of four computer engineers from Cairo university have taken it upon themselves to find out a recipe for the perfect airline company by answering the paramount question “What makes airline customers satisfied?”. The question is posed as both a data analysis problem and a machine learning problem that together answer it via exploratory analytics, association rule mining and predictive models.

Our approach to said problem utilized the following pipeline

The following is the implied folder structure:

.
├── DataFiles
│ ├── airline-train.csv
│ └── airline-val.csv
├── DataPreparation
│ ├── DataPreparation.py
│ └── Visualization.ipynb
└── ModelPipelines
├── Apriori
│ ├── Apriori.ipynb
│ └── rules.txt
├── NaiveBayes
│ ├── NaiveBayes.ipynb
│ └── NaiveBayes.py
├── RandomForest
│ ├── RandomForest.ipynb
│ └── RandomForest.py
└── SVC
├── SVC.ipynb
└── SVC.py

pip install requirements.txt
# To run any stage of the pipeline, consider the stage's folder. There will always be a demonstration notebook.

We have also run the project on the cloud via Azure ML studio. You may contact the developers if you find any issue in that.

We used Kaggle's Airline satisfaction dataset as shown below

Our data preparation module was implemented using Pandas and PySpark and supports the following:

• Reading a specific split of the data (training or validation)
• Reading specific column types from the data (numerical, ordinal or categorical)
• Frequency Encoding for categorical features
• Dropping missing values
• Imputing numerical outliers Alternatives for the function were implemented as well in case any model required further special preprocessing.

Instead of querying the dataset for specific facts, we priotized that it should tell us all the facts. In other words, we have let the data speak for itself and for that we designed the following analysis workflow

In the following we will take you through a cursory glance of the workflow, the set of insights that each visual corresponds to (over 50 insights in total) can be found in the demonstration notebook or the report along with full versions of the visuals and tables.

In this we study if there is any imbalance among the satisfaction levels of people involved in the dataset.

This provides a description, type and possible values for each feature.

In this, we analyze each feature for missing data.

For each type of feature, we consider measures of central tendency and spread. This is an example for numerical features.

Here we analyze the distributions of each feature to look for any special patterns.

This provides the same analysis as above, but per class.

We have used the Chi-square test of independence which tests if there is a relationship between two categorical variables. In particular, we have that

$H_0$: The two categorical variables are independent.

$H_1$: The two categorical variables are dependent.

Here, we set $\alpha = 0.05$ and hence, if the p-value for the test done on two variables is less than $0.05$, we reject the null hypothesis and conclude that the two variables are dependent.

We have used Pearson's correlation ratio to find associations between all possible numerical and categorical features

We have use Spearman's to find monotonic associations between ordinal variables

We have use Pearson's to study linear correlation between numerical variables

Related to dependence is also testing the Naive bayes assumption. We hae provided an automated way for that with the sample output:

As expected, the Naive Bayes assumption does not hold. In particular, we have that $$P(x_1, x_2, ...|C_1=0)=0.16$$ as computed numerically using the definition of the probability. Meanwhile, applying the Naive Bayes assumption we have that $$P(x_1, x_2, ...|C_1=0)=P(x_1|C_1=0)P(x_2|C_1=0)...=0.08$$ which is different from the correct probability.

Likewise, for the class $C_1=1$ we have that $$P(x_1, x_2, ...|C_1=1)=0.07$$ but $$P(x_1, x_2, ...|C_1=1)=P(x_1|C_1=1)P(x_2|C_1=1)...=0.04$$ which is different from the correct probability.

This does not stop up from using the model as research has demonstrated that it can be robust to the assumption.

We prepared box plots to study the distribution of the numerical features for each class in the target

Then we started analyzing all pairs to see if separability gets better

Then we considered all trios

We then sought interaction with all possible numerical features. The plot is much longer in the notebook.

And interaction among all possible categorical features. You must have enough RAM to see the full version in the notebook.

The abundance of categorical features and their significance as shown above has inspired considering Naive Bayes and Random Forest as predictive models. We later follow up with an SVM model as ordinal features can be assumed as numerical as well. But before employing such models we considered topping off our exploratory data analytics with association rule learning.

For this, numerical features (excluding extremely skewed ones) were converted to categorical ones by binning and then all the categorical features were one-hot encoded.

The following shows a sample of the strongest rules found:

Where the corresponding graphic over all strong rules is

This is color plot of support against the confidence where the color represents lift and the number refers to a rule in the table.

For Naive Bayes, we started with preprocessing the data by converting the string categories to integers and bucketizing the four numerical variables based on the 10 percentiles (10%, 20%, 30%,...) so that they can be treated similar to categorical variables. We followed by implementing NaiveBayes on PySpark from scratch:

Since it holds by Bayes rule: $$P(C \mid A=(a_1,a_2,\ldots,a_M)) = \frac{P(A=(a_1,a_2,\ldots,a_M) \mid C)P(C)}{P(A=(a_1,a_2,\ldots,a_M))}$$

Since it holds by the naive assumption: $$P(A=(a_1,a_2,\ldots,a_M) \mid C) = \prod_{m=1}^{M} P(a_m \mid C)$$

By the constant denominator in Bayes: $$P(C_i \mid A)=(a_1,a_2,\ldots,a_M) \propto P(A=(a_1,a_2,\ldots,a_M) \mid C)P(C)$$

Hence, the most likely class is given by: $$C = \max_{1 \leq i \leq K} { P(C_i) \prod_{m=1}^{M} P(a_m \mid C_i)}$$

where $P(C_i)$ and $P(a_m \mid C_i)$ are easilt computed by counting.

This has yields an accuracy of $89.1%$ in terms of predicting customer satisfaction.

We also considered initiating a Random Forest model which did not further require any special processing (beyond NB). Luckily, PySpark’s RandomForest inherently supports both categorical and numerical features after applying the model the perceived accuracy on the validation set was $96%$.

We analyzed the average feature importance set by trees in the forest to yield

We topped off with a linear SVM model and hyperparameter search but results were not as significant as the random forest.

The totality of the analyses above have led us conclude the following:

• There is a lack of satisfaction in airline travel experience (imbalance)
• Such lack is focused on economy travelers
• Wifi Service, Entertainment and OnlineBoarding are key determinants of satisfaction
• Comfort and ease of booking also matter
• Distance and delays seem to have a less adverse effect

Essam

Halahamdy22

MUHAMMAD SAAD

Noran Hany

We have utilized Notion for progress tracking and task assignment among the team.