Enhancing Debugging Capabilities for cf-autotick-bot

[0xbe7a]

Hello everyone,

We're aware that when the cf-autotick-bot encounters a failure, it can sometimes be a challenging task to debug. The failure can either be silent, such as when the update strategies fail to find a new release, or it can be explicit, like when the feedstock couldn't be updated. In both scenarios, the debugging process can be quite complex due to the scattered information and the intricate behavior of the bot.

Here are a few examples of issues i am describing:

• Version upgrade bot failures #1583
• boost-cpp missing version updates #1588
• Autotick bot not running after changes in multi-source multi-output feedstock #1540
• python-lmdb missing update PRs #1498
• Missing libgsasl 1.10.0 update PR #1404

To improve the debugging experience, we're considering the development of a dedicated CLI tool, entry point, or public web app. This tool would utilize the same update logic as the cf-autotick-bot but would be specifically designed to assist maintainers in running the logic for just a single feedstock. The goal is to generate comprehensive debug information to simplify the debugging process, identify failures, and facilitate the application of mitigations by the maintainers themselves.

At present, debugging the bot's logic isn't straightforward, as it's designed to run continuously over all-packages using the cf-graph, rather than in a one-shot fashion.

To address this, we're excited to announce that one of our talented interns will be spearheading this project. The intern will be closely mentored by @xhochy and myself throughout this process. We'll provide guidance and support to ensure the development of an effective solution that will enhance the debugging process.

Please feel free to share any thoughts or suggestions you may have.

[beckermr]

SGTM! We do try to expose stack traces on the status page but they are not always helpful. I think the places to focus are:

1. Debug info on why the bot did or did not skip a given feedstock in a migration.
2. Ability to apply the migration, rerender, and then run the solver checks locally.
3. Simpler ways to debug which upstream version / url the bot finds.