MTR flag to mark tests as incompatible with macOS

[DaveGosselin-MariaDB]

Introduces a new MTR include, not_mac.inc, which when included at the top of a test, prevents that test from running on macOS.

There are many tests that don't work with macOS presently and they are not excluded in this commit; however, one is, as an example. The remaining tests will be fixed or marked as incompatible in future commits.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[DaveGosselin-MariaDB]

Evidently there is something missing from this patch.

[DaveGosselin-MariaDB]

OK it works now.

[tonychen2001]

Minor suggestion:
This is redundant as including not_mac.inc in the line below already implies this.
Perhaps a brief explanation of why this test doesn't work on macOS as a comment or in the commit description would be better.

Apart from that, looks good.

[DaveGosselin-MariaDB]

Hi Tony, thank you for the suggestion. I haven't (yet) investigated why this test doesn't work on macOS, but I will. Just not for this commit; its purpose is to demonstrate how to use not_mac.inc.

[grooverdan]

Is mac generating different results than unix? Is so might need:

$ git diff mysql-test/
diff --git a/mysql-test/include/platform.combinations b/mysql-test/include/platform.combinations
index 4f0660b7a40..672f78e1fc5 100644
--- a/mysql-test/include/platform.combinations
+++ b/mysql-test/include/platform.combinations
@@ -4,3 +4,4 @@
 
 [unix]
 
+[mac]
diff --git a/mysql-test/suite.pm b/mysql-test/suite.pm
index f30cc5ec431..4800c1e7dc6 100644
--- a/mysql-test/suite.pm
+++ b/mysql-test/suite.pm
@@ -18,11 +18,13 @@ sub skip_combinations {
 
   # don't run tests for the wrong platform
   if (IS_WINDOWS) {
-    $skip{'include/platform.combinations'} = [ 'aix', 'unix' ];
+    $skip{'include/platform.combinations'} = [ 'aix', 'unix', 'mac' ];
   } elsif (IS_AIX) {
-    $skip{'include/platform.combinations'} = [ 'win', 'unix' ];
-  } else {
-    $skip{'include/platform.combinations'} = [ 'aix', 'win' ];
+    $skip{'include/platform.combinations'} = [ 'win', 'unix', 'mac' ];
+  } elsif (IS_MAC) {
+    $skip{'include/platform.combinations'} = [ 'aix', 'win', 'unix' ];
+  } else { # unix
+    $skip{'include/platform.combinations'} = [ 'aix', 'win', 'mac' ];
   }
 
   if ( $::opt_ps_protocol ) {

There are a few platform rdiff tests that would need recording on this:

$ find mysql-test/ -name \*rdiff | grep -E 'aix|win|unix'
mysql-test/main/mysqld--help,win.rdiff
mysql-test/main/mysqldump-system,win.rdiff
mysql-test/main/mysqld--help,aix.rdiff
mysql-test/suite/sys_vars/r/sysvars_server_notembedded,aix.rdiff

Otherwise including in a lower branch like 10.4 would also some use while tests can fixed/improved.

[vuvova]

would be great to see a test that needs include/not_mac.inc, with an explanation of what's so special in mac that the test doesn't work there. Not file name case sensitivity, not mac's CrashReporter, but really, just "mac"

[DaveGosselin-MariaDB]

would be great to see a test that needs include/not_mac.inc, with an explanation of what's so special in mac that the test doesn't work there. Not file name case sensitivity, not mac's CrashReporter, but really, just "mac"

@vuvova I devised this mechanism as a way for us to get the mac buildbot working without fixing every test first. At the same time, this would prevent new test failures from creeping in because we could make the mac builder "required" when pushing commits, preventing deliveries from introducing new mac test failures. What you're asking for seems like a diagnosis of each mac test failure that we know about. At this point, that is time consuming which meanwhile allows for more failures to appear. Is such a pragmatic approach not desired, and if so why not? In short, it would be nice to plug the holes in the hull before bailing the water out.

[DaveGosselin-MariaDB]

Hi @vuvova. A comment from @abarkov on MDEV-33652 highlights the need for this exclusion mechanism to be included sooner rather than later. In short, he'd like to wait for another fix to merge to 11.5 so that he can fix a failing macOS test in another way. (My original fix is here). I have no problem with that, but in the meantime this test will continue to fail on macOS. If the exclusion mechanism was in place, then we could leave a comment for Alex on his ticket to re-enable this failing test as part of his delivery.

[abarkov]

Hi @DaveGosselin-MariaDB , in the meanwhile the test could be fixed not to use upper case letters in the object names.

[vuvova]

@DaveGosselin-MariaDB my point was that this particular test, and, I suspect many mac-specific failures could be covered by include/have_case_insensitive_fs.inc. That is, the question was what tests fail not just on case insensitive file system but on mac because of its inherent macness ?

[DaveGosselin-MariaDB]

Hi @vuvova , ah I understand now. There's nothing about inherent 'macness' that breaks this test. The test in question could be fixed either by @abarkov 's suggestion or yours (to use include/have_case_insensitive_fs.inc). I will fix it and others having case-sensitivity issues (full list on MDEV-33616 ) in either of those two ways, as context permits. Thanks for the feedback.