This is the project page for Model Assertions for Monitoring and Improving Machine Learning.

Please read the paper for full technical details.

pip install model_assertions

We provide two examples.

The first example (Tabular.ipynb) shows an example of predicting house prices from features. This example trains a lienar model to predict the house price.

We define a model assertion that asserts the predicted house price should be positive. While seemingly simple, the predictions violate this assertion!

The second example (Consistency.ipynb) shows an example of predcting people in a TV show and several attributes of the person (gender and hair color). In this example, we assume that the predictions are already provided.

This example shows how to use the attribute- and time-consistency APIs. It asserts that the same person in the same scene should have the same gender and hair color. It also asserts that a person in the same scene should appear across consecutive frames without gaps.

If you find this project useful, please cite us at

@article{kang2020model,
  title={Model assertions for monitoring and improving ML model},
  author={Kang, Daniel and Raghavan, Deepti and Bailis, Peter and Zaharia, Matei},
  journal={MLSys},
  year={2020}
}

and contact us if you deploy model assertions!