Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stats generation job fails entirely on deadlock detection #583

Open
lladdy opened this issue May 20, 2023 · 2 comments
Open

Stats generation job fails entirely on deadlock detection #583

lladdy opened this issue May 20, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@lladdy
Copy link
Contributor

lladdy commented May 20, 2023

Occasionally, a deadlock will occur when attempting to lock a competition participation.
Instead of the entire stats gen job falling over, it should at minimum skip the competition participation in question.

More ideally it should wait and then retry whatever caused the deadlock, with some max retry attempt.

@lladdy lladdy added the bug Something isn't working label May 20, 2023
@lladdy
Copy link
Contributor Author

lladdy commented Jul 9, 2023

This has been improved with an update but still results in bots not having their stats updated for up to ~a week.

@lladdy
Copy link
Contributor Author

lladdy commented Aug 31, 2023

Relevant stats job code is here, for anyone interested in trying to refine it:
https://github.com/aiarena/aiarena-web/blob/staging/aiarena/core/management/commands/generatestats.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant