Having doo files part of the git index prevents agile development

[MikaelMayer]

We should remove the doo files from the index. They will cause conflict at every pull request if that pull request changes the standard library. Moreover, they don't build the same on every machine currently.
Committing generated files is not a good practice in general. We do this for Source/DafnyCode/GeneratedFromDafny.cs because this file is needed for DafnyCore to work, as per our bootstrap strategy. But even though we could depend on the standard library files, we won't use the doo file for that, we will likely integrate the standard library sources inside GeneratedFromDafny.cs under their compiled version.

So really, doo files should be removed from the index.

Suggested change

• Remove all the doo files from the index.
• Build them on demand before compiling Dafny.exe

[robin-aws]

This is all true, with an additional technical challenge: although the doo files (and similarly the some of the runtime artifacts built using Dafny) are not actually needed to execute most of the core Dafny tool, they are ultimately packaged with them, so it still creates a packaging circular dependency. This was the motivation for splitting up the projects so that we had DafnyDriver, which transitively contains nearly all the dafny CLI functionality, and DafnyPipeline, which embeds assets like the runtimes and standard library doo files. That makes it at least possible to build the former and then use it to build the latter.

The one downside is that I have not been able to find a way to make that process magically work with one execution of dotnet build Dafny.sln: the dotnet CLI sometimes locks up if you try to make a recursive call to build. It seems like there's no way around making building Dafny a two-step process, but that seems like a worthwhile trade off to fix this issue. It will also us to stop checking in the growing compiled contents of the runtimes (see #4877)