Sprig Runbook

References

Grafana Dashboard: telemetry.hackclub.com
Vercel Project: hackclub/vercel
Sprig Grafana Alerts sprig @ grafana alerts
Sprig Grafana Dashboard sprig @ grafana

Rollback to the latest stable deployment

Go to vercel hackclub/sprig project and click the instant rollback button.

Then select the previous deployment.

Database operations fail or take too long

To see what database operations are failing, open the sprig dashboard in grafana here and look under the 'Endpoint latencies over time' visualization or the 'Endpoint failures' visualizations.
Open the Vercel Logs for Sprig and checkout logs with level 'Error' or 'Warning'.

Attempt to trigger a failing operation

If the failing database operation is any of the following, open the live logs on vercel while doing one of the following.
- api_games_save, reproduce by opening one of your games in the editor, make an edit
- api_games_delete, reproduce by deleting one of the games in your accout from the dashboard
- api_games_rename, reproduce by opening one of your games in the editor, edit the games's name
- api_thumbnail, reproduce by opening the sprig gallery
- api_auth_email-login-code or api_auth_submit-code, reproduce by logging into your account

Sprig stops reporting to grafana

Go to environment variables and the check if the 'GRAPHITE_HOST' is correctly set to telemetry.hackclub.com

An error you are unsure about

Open the Grafana Dashboard and look at the visualizations.
Open the Vercel Logs and look at logs with log level 'Warning' or 'Error'
Alternatively, open the developer console console on the sprig website and look at the client-side logs. Problematic situations should most likely be apparent.