This project addresses the challenge of hospital readmissions, focusing on diabetic patients, a significant issue that has a great impact on healthcare costs. It was developed for the "Machine Learning" course from the MSc in Data Science and Advanced Analytics, at NOVA IMS.
The project resolved on developing two models:
(Note: only models from scikit-learn could be used for this project.)
The dataset consists of 29 features and 2 target features (one for binary, one for multiclass). It includes crucial features such as patient identifiers, demographic information, health-related details, historical healthcare utilization, admission specifics, vital signs, discharge information, length of hospital stay, lab tests, procedures, medications, and diagnostic codes.
For more detailed explanation of attributes, click here.
In the Datasets folder, there are two available datasets, the train (model training and validation) and test data (performance of trained models on unseen data).
Evaluation Metrics: Utilized F1 score as the primary metric for model evaluation, considering its significance in identifying incorrectly classified cases.
| Model | F1 | Mean Accuracy | Train Score | Test Score |
|---|---|---|---|---|
| Random Forest | 0.70 | 0.760 | 0.789 | 0.695 |
| Hist Gradient Boosting | 0.66 | 0.685 | 0.693 | 0.658 |
| Stacking (NN + ET) | 0.64 | 0.684 | 0.690 | 0.644 |
Conclusion: For model selection, we opted for a Stacking model comprising a Neural Network classifier and an Extra Trees classifier due to its effectiveness in achieving balanced performance across metrics.
Evaluation Metrics: Utilized Macro-Averaging for fair evaluation of all classes, considering equal weight for each class.
| Model | F1 | Mean Accuracy | Train Score | Test Score |
|---|---|---|---|---|
| AdaBoost | 0.60 | 0.536 | 0.597 | 0.544 |
| Random Forest | 0.59 | 0.530 | 0.672 | 0.590 |
Conclusion: Adaptive Boosting slightly outperformed Random Forest with a mean accuracy score of 0.5970 against 0.59059.
Based on the analysis conducted for both binary and multiclass classification tasks, we arrived at some significant conclusions. Firstly, our chosen models demonstrated promising predictive capabilities, with probabilities of patient readmission calculated at 8% for multiclass classification and 40% for binary classification. These probabilities were derived from a careful assessment of true positives and false positives within relevant classes.
Moreover, the features selected for model inclusion align well with our initial research objectives, emphasizing factors such as hospital length of stay, medication, and associated diagnoses. Notably, the 'recurrency' feature emerged as particularly influential, reflecting the heightened risk associated with repeated patient admissions.
Despite these promising outcomes, our study encountered certain limitations. We are almost certain, that we can achive better scores by engineering better features from the existing dataset. We faced some challenges in picking the right tools and techniques for our study, and the process of analyzing the data was time-consuming. Despite these challenges, our models performed similarly well.