Switch to using unittest.main and custom runner for unittest design

[eleanorjboyd]

Currently the steps to run unittest tests are broken down into individually calling:

loader = unittest.TestLoader()
tailor: unittest.TestSuite = loader.loadTestsFromNames(test_ids)
result: UnittestTestResult = runner.run(tailor)  # type: ignore

This limits our ability to run tests as similar to how they are run in the command line and to move all heavy lifting over to unittest.

Instead we should use unittest.main(module='__main__', testRunner=None, ... with a custom testRunner that is already written to run unittest. Therefore we can more easily just run any args provided and then also gives users more flexibility. This change is similar in design to that of adding Django support with the use of the testRunner argument.

Looking for community input on useful this change is. Additionally the issues I think it would help with are as follows:

• support all unittest args: Unittest args doesn't support -t flag  #5721

• (unsure about this one but might fix) load_test protocol allow usage of unittest's load_tests protocol in python unit tests #12063

• Python unittest runner doesn't respect arguments to pass to unittest runner #20425

• supporting -s Backslashes doesn't work when specifying the paths with python.testing.unittestArgs #11670

• support -s and -t Incorrect unit test paths in side panel when using both -s and -t parameters #21267

[github-actions]

Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

[brettcannon]

Thank you for submitting your feature request and everyone who considered it! Unfortunately, this issue did not receive enough votes over the allotted time, and so we are closing the issue.

[eleanorjboyd]

This will be required for switching to using Django so it will be a step as we implement that change.

[carltongibson]

This is basically the main missing element in the Django support.

Django being quite a large Python web framework, it seems in everyone's interest to add this.

I'm not sure if anyone in the Django community was aware there was an issue to show support for, or that there was a time limit for it to be voted on, but I'm glad it's been reopened. I don't know if Code has any community outreach but that would have been handy: there are lots of Django folks using it.

👍

[iamoeg]

Also a Django user, and I second @carltongibson's comment. Please push this forward.

[eleanorjboyd]

Hi! This is a priority for us but is blocked on some work we have to do related to custom arguments which will enable setting the django variables required to use manage.py. The main issue which tracks this work and our proposed design can be found here: #22206 but it is based off a historical issue #73 with the request. It is the most highly voted feature on a repo so it is a priority for us and we are working to get there. Thanks!