Exit 0 Status Code on Report Failure

[JackWink]

We've noticed that pronto exits with a code of 0 when it fails to report to Github. --exit-code seems to only report non-0 exits when there is a lint error, not an internal failure.

Right now we're working around it by capturing stderr and testing for it to be empty:

RESULT=$([... env vars] bundle exec pronto \
  -f github_pr \
  -f github_status \
  -c origin/master 2>&1)
test -z "$(echo $RESULT | grep "Pronto errored: ")"

But it would be nice if there were an additional flag to specify an exit code for reporting issues or internal pronto errors.

[aadityataparia]

Same problem. We want to exit with non-0 status when there in a failure (not lint error) and 0 exit code when there are only lint errors.

[jbielick]

Adding another example:

We run pronto in CI like so:

pronto run --exit-code -f text github_pr github_status -c origin/master

The only runner we're using is pronto-rubocop.

What we expected to happen:

1. We'd like PR comments (github_pr achieves this)
2. We'd like a status check (github_status achieves this)
3. We'd like the pronto run output to be in the CI log (text achieves this)
4. We'd like the CI job to fail if there are any pronto offenses reported.

What actually happens:

Items 1-3 are all fine.

Item #4 does not happen. Pronto exits with code 0 when the github formatters can't find a Pull Request for the current commit. I like that the github formatters effectively noop, but now CI passes! The underlying issue is this: developer commits, pushes branch to remote. CI runs, pronto finds an issue, but the build is green. No comments can be made. A PR is later opened and the PR status checks are green, no comments are ever added by pronto and the PR merges without any feedback from pronto except the stdout like @JackWink is describing.

[pbstriker38]

Is there any traction on this issue?