Add end-to-end test framework inspired from iOS interactive tests #8390
Conversation
This brings interactive tests from iOS and allows them to be run in any environment. Specifically, given a set of `Game` classes under `Tests/Interactive/Tests`, we can run that `Game` instance in a test harness under DesktopGL (Mac/Windows)/iOS (Android coming next). - Lots of reshuffling from Tests/Interactive/iOS -> Common/Tests - Also added doc to run the tests in simulator/on-device. - New TestRunners (for now DesktopGL on Mac/Windows and iOS)- will add Android next - DesktopGL only allows running one test per run, due to STA/SDL Window event loop. - Perhaps consider NUnit test that launches the DesktopGL in a separate process with the correct command-line parameter? - Added doc in Tests/Interactive/README.md on how to run the test runners. - Allows 'automatic' running of InteractiveTests too (i.e. automatically run one test, allow test to signal 'finish' and exit) - i.e. provided a `TestGame` that implements `Game` and allows a test author to signal when the test is done. (i.e. either by clicking the Exit button or the test can run automatically, then signal Exit)
|
@dellis1972 @SimonDarksideJ Please take a look. I would like an initial pass to see if this is even worth it. This also doubles as a test tutorial for newbies to quickly test DesktopGL/iOS etc (especially specific things like alpha blend test or render target test as these are common questions in discord/forums). It's also pretty well aligned with the bounty to add tutorial(s) although I am not partaking in that. |
|
To start, please look at README.md which is a good starting point. |
| } | ||
|
|
||
| /// <summary>Output an error message to the console.</summary> | ||
| public static void E(string message) |
There was a problem hiding this comment.
I think I mentioned this on the other PR.
It might be better to have actual descriptive method names like LogError or Error. Same for the C method. Looking at the code which is calling it , it is not obvious what the method does GameDebug.C ("foo") could be doing anything, so you have to go look.
In this case C might be better being called Log or Debug.
There was a problem hiding this comment.
NVM, this is the *test* specific logging, not `GraphicsDebug` - I will make this change as the test doesn't require conciseness - I favor readability to conciseness in the test. (Edited comment )
Before edit: I was going for the minimal [Android-y](https://developer.android.com/reference/android/util/Log) `Log` that are concise and help avoid cognitive overload when reading a bunch of the logging statements.
If the xmldoc comment doesn't help (on hover for example), I am okay with changing it to be a bit longer like you suggest. But I wanted to provide you a rationale first.
It also helps with using GD=GraphicsDebug; GD.E("..."); etc which makes it super concise.
There was a problem hiding this comment.
I understand the rationale. I would favour consistencey though, most of the logging in the repo uses things like Debug.WriteLine or Log.Verbose etc (see https://github.com/MonoGame/MonoGame/blob/develop/MonoGame.Framework/Platform/Android/MonoGameAndroidGameView.cs#L778). Keeping code consitent with other uses within the repo makes it easier to people to understand the code as a whole. This is my preference, we can see what other @MonoGame/maintainers think.
There was a problem hiding this comment.
I would agree with @dellis1972 , as this is the standard used by the rest of MonoGame.
There was a problem hiding this comment.
Done PTAL, thanks!
| private static readonly Dictionary<string, MessageCount> MESSAGES_COUNTS_ = new(); | ||
|
|
||
| /// <summary>Use this to output spammy messages.</summary> | ||
| public static void Spam(string message, int maxNumTimes = 10) |
There was a problem hiding this comment.
Pinging :) Could you please take a look? Thanks!
This brings interactive tests from iOS and allows them to be run in any environment.
Specifically, given a set of
Gameclasses underTests/Interactive/Tests, we can run thatGameinstance in a test harness under DesktopGL (Mac/Windows)/iOS (Android coming next).TestGamethat implementsGameand allows a test author to signal when the test is done. (i.e. either by clicking the Exit button or the test can run automatically, then signal Exit)