Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pretty print model summaries #25

Closed
wants to merge 23 commits into from
Closed

Pretty print model summaries #25

wants to merge 23 commits into from

Conversation

tovbinm
Copy link
Collaborator

@tovbinm tovbinm commented Jun 21, 2018

  • Added pretty summary functionality for OP models - model.summaryPretty (example output below), which includes selected model info, model evaluation results and feature correlations/contributions/cramersV values.
  • Added a handful of methods to ModelInsights to retrieve information about validation and evaluation results
  • Modified and add some enums
  • Added tests

Example output model.summaryPretty() function:

Evaluated LogisticRegression model using Cross Validation and area under PR metric.
Evaluated 2 LogisticRegression models with area under PR metric between [0.25, 0.25].
+--------------------------------------------------------------------------------------------+
|                            Selected Model - LogisticRegression                             |
+--------------------------------------------------------------------------------------------+
| Model Param      | Value                                                                   |
+------------------+-------------------------------------------------------------------------+
| aggregationDepth | 2                                                                       |
| area under PR    | 0.25                                                                    |
| elasticNetParam  | 0.1                                                                     |
| family           | auto                                                                    |
| featuresCol      | age-gender-height-survived-weight_6-stagesApplied_OPVector_000000000094 |
| fitIntercept     | true                                                                    |
| labelCol         | survived_1-stagesApplied_RealNN_000000000093                            |
| maxIter          | 50                                                                      |
| modelType        | LogisticRegression                                                      |
| name             | logreg_69488384749f_0                                                   |
| predictionCol    | prediction                                                              |
| probabilityCol   | probability                                                             |
| rawPredictionCol | rawPrediction                                                           |
| regParam         | 0.01                                                                    |
| standardization  | true                                                                    |
| threshold        | 0.5                                                                     |
| tol              | 1.0E-6                                                                  |
| uid              | logreg_69488384749f                                                     |
+------------------+-------------------------------------------------------------------------+
+-------------------------------------------------------+
|               Model Evaluation Metrics                |
+-------------------------------------------------------+
| Metric Name | Training Set Value | Hold Out Set Value |
+-------------+--------------------+--------------------+
| AuPR        | 1.0                | 0.0                |
| AuROC       | 1.0                | 0.0                |
| Error       | 0.0                | 0.0                |
| F1          | 1.0                | 0.0                |
| FN          | 0.0                | 0.0                |
| FP          | 0.0                | 0.0                |
| Precision   | 1.0                | 0.0                |
| Recall      | 1.0                | 0.0                |
| TN          | 4.0                | 1.0                |
| TP          | 1.0                | 0.0                |
+-------------+--------------------+--------------------+
+-------------------------------------------------+
|               Top Model Insights                |
+-------------------------------------------------+
| Top Positive Correlations |   Correlation Value |
+---------------------------+---------------------+
| weight(weight = null)     |  1.0000000000000002 |
| gender(gender = null)     |  1.0000000000000002 |
| age(age = null)           |  0.6123724356957946 |
| weight                    |                 0.0 |
| age                       |                 0.0 |
| height                    | -0.5292379183351017 |
| gender(gender = other)    | -1.0000000000000002 |
+---------------------------+---------------------+
+-------------------------------------------------+
| Top Contributions      |     Contribution Value |
+------------------------+------------------------+
| gender(gender = other) |    -2.5506146558069944 |
| weight(weight = null)  |     2.5506097053715098 |
| gender(gender = null)  |     2.5506097053715098 |
| age(age = null)        |     0.8801282009379419 |
| height                 | -0.0029105396140691126 |
| age                    |  0.0015760812856771072 |
| weight                 |     3.3953008369728E-4 |
+------------------------+------------------------+

@tovbinm tovbinm requested review from snabar and kinfaikan June 21, 2018 06:47
@tovbinm tovbinm changed the title Pretty print model summaries WIP - Pretty print model summaries Jun 21, 2018
@tovbinm tovbinm requested a review from leahmcguire June 21, 2018 07:01
@tovbinm tovbinm changed the title WIP - Pretty print model summaries Pretty print model summaries Jun 26, 2018
@tovbinm tovbinm requested a review from Jauntbox June 26, 2018 03:55
@tovbinm
Copy link
Collaborator Author

tovbinm commented Jun 30, 2018

Moved to internal repo.

@tovbinm tovbinm closed this Jun 30, 2018
@tovbinm tovbinm deleted the mt/pretty-prints branch June 30, 2018 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant