I'm kinda new to making GitHub Actions, so I'm not actually sure how I can test this (either locally or by publishing the action under a new name from my fork of this repo)

I figured out how to publish this as a test action, but it doesn't seem to work. I suspect that publishing the annotations on a commit ID in the target repo causes them to just not get associated with anything, and thus lost.

I would suggest not merging this PR until it can be reworked by someone who understands the GitHub APIs better than I :)

I suspect that publishing the annotations on a commit ID in the target repo causes them to just not get associated with anything, and thus lost.

Yes that's correct, the same thing happened to me while I was working on c707e6d.

I would suggest not merging this PR until it can be reworked by someone who understands the GitHub APIs better than I :)

I took a look at other GitHub Actions but so far I have yet to find a working solution 😞