Support xfail/skip of optional, format, next tests

[handrews]

So... I recall starting this idea while coming down with COVID, but didn't realize I'd mostly gotten it working. So here's my last leftover old branch :-) The idea here is to document the optional/format/draft-next support and enable it in PR test runs (which I'll post as a separate PR) in order to get the coverage numbers for PRs like #93 (draft-next contains, etc., which currently fails the coverage checks because the new code is only covered by draft-next tests).

Maintaining this data manually is impractical, so of course I had to write an elaborate system to automate it 😅 @marksparkza I will not be bothered if you decide this is more trouble than it's worth, but it does work. It's also nice when looking at implementations to see what (if any) optional features and formats are supported in case any of that is needed for someone's use case. But it is kind of a lot.

Commit message:

---

This adds infrastructure to generate a JSON file flagging which optional, format, and/or draft-next tests are expected to fail ("xfail") or have a bug that requires skipping them ("skip"). The file includes both the status to annotate the test, as well as the reason.

The --testsuite-generate-status option will run all tests and update the JSON file based on the current failures. It will leave in place existing entries for failures so as not to overwrite and custom statuses or reasons. It will remove entries for tests that now pass.

Failing cases are set to "xfail" with an appropriate generic reason (format, optional, or draft-next, in that order of precedence). To skip a test, or provide a custom reason, edit the file manually.

This commit also includes documentation in in the CONTRIBUTING file for test options I added a while back but apparently forgot to document.

[codecov-commenter]

Codecov Report

Patch coverage has no change and project coverage change: +0.44 🎉

Comparison is base (0b9b299) 92.17% compared to head (07e52b0) 92.62%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
+ Coverage   92.17%   92.62%   +0.44%     
==========================================
  Files          21       23       +2     
  Lines        2006     2033      +27     
  Branches      427      428       +1     
==========================================
+ Hits         1849     1883      +34     
+ Misses        100       97       -3     
+ Partials       57       53       -4     

see 11 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[marksparkza]

This is brilliant! Great to be able to see exactly which optional features / formats are supported or not.

I've just tweaked the docs a bit and removed a trailing blank line from suite_status.json - it shouldn't be added by json.dump and when I ran pytest --testsuite-generate-status it removed the trailing blank.