Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with multi-backend-testing #5533

Open
MikaelMayer opened this issue Jun 5, 2024 · 0 comments
Open

Issue with multi-backend-testing #5533

MikaelMayer opened this issue Jun 5, 2024 · 0 comments
Labels
kind: language development speed Slows down development of Dafny the language, flaky tests priority: not yet Will reconsider working on this when we're looking for work

Comments

@MikaelMayer
Copy link
Member

Currently, running the following command has a problematic side-effect

DAFNY_INTEGRATION_TESTS_IN_PROCESS=true
DAFNY_INTEGRATION_TESTS_MODE=only-multi-backend
DAFNY_INTEGRATION_TESTS_ONLY_COMPILERS=dfy
DAFNY_INTEGRATION_TESTS_ROOT_DIR=..../dafny/Source/IntegrationTests
DAFNY_INTEGRATION_TESTS_UPDATE_EXPECT_FILE=true
dotnet test Source/IntegrationTests/IntegrationTests.csproj

Indeed, it starts creating .dfy.dfy.check files for EVERY file because none of them pass the ResolvedDesugaredExecutableDafny backend by throwing FeatureNotImplementedException(Feature.RunAllText), which is just a placeholder for "this backend is exected to fail here no need to report it".
These files all contain the following:

// CHECK-L: (0,-1): Error: Feature not supported for this compilation target: The /runAllTests option

It's for every file because this gets called for every file
_out14 = D2DPrettyPrinter.__default.PrettyPrint(p);
which launches the following:
Microsoft.Dafny.Compilers.WrapException.Throw();
in any case.

While is is technically correct, at the moment there is not a single integration test file that will not emit that check file, and these check files are useless for now.
Our CI does not care as it's not set up to generate files. However, locally, if we are set up to generate those files, they will start appearing and it's a bit problematic.

The reason why ResolvedDesugaredExecutableDafny is marked as stable is to be able to catch any other exception than the one mentioned above.
But now I realize it's an issue. Perhaps I should just deactivate the emitting of check files for that particular compiler, what do you think @robin-aws ?

@MikaelMayer MikaelMayer added the kind: language development speed Slows down development of Dafny the language, flaky tests label Jun 5, 2024
@stefan-aws stefan-aws added priority: not yet Will reconsider working on this when we're looking for work and removed priority: unknown labels Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: language development speed Slows down development of Dafny the language, flaky tests priority: not yet Will reconsider working on this when we're looking for work
Projects
None yet
Development

No branches or pull requests

2 participants