Introduce an async type DynamicStack that can be used to prevent stac…

[keyboardDrummer]

Introduce an async type DynamicStack that can be used to prevent stack overflows in recursive code.

Documentation: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-7.0/task-types
More info: https://devblogs.microsoft.com/premier-developer/dissecting-the-async-methods-in-c/?utm_source=pocket_mylist

[robin-aws]

Based on a quick skim, this is a cool idea with promise. I naturally worry about the overhead of using await/async for what is actually a sequential algorithm, but I could easily be convinced with benchmarking data that C#/CLR is clever enough about optimizing that case (and this kind of processing is not where I'd worry about minor inefficiencies).

Another alternative is to use the visitor pattern more. A lot of AST processing is using that, even though there is surely room to improve the UX. Can we experiment with how that would look here? (perhaps on a smaller subset of the PrintExpression code as that's a lot of lines to change :)). On the flip side, if this approach works well it might be worth migrating existing visitor-based code to it eventually instead.

[cpitclaudel]

Another alternative is to use the visitor pattern more.

Does the visitor pattern help with recursion depth?

[keyboardDrummer]

I naturally worry about the overhead of using await/async for what is actually a sequential algorithm

It's still executed sequentially in this PR. There are no tasks or threads involved. For every method with async DynamicStack, the .NET compiler will split its code up into parts that are separated by the await operator. A class is generated for the method, which I'll call the 'frame class', that contains a MoveNext() operator which executes the method part that the class is currently at and then increments an integer to move to the next part.

When an async DynamicStack method is called, an instance of its frame class is created (performance similar to new new object()) and that instance is placed on a thread specific execution stack. When the await operator is used inside a async DynamicStack, the current method's frame class is placed on the execution stack and the remaining code is not yet executed.

Calling Run() on a dynamic stack will pop items from the stack, calling MoveNext() after each pop, until the stack is empty. Initially there's only a single item on the stack when Run() is called. During MoveNext() calls more items are added to it until eventually the algorithm finishes.

You can find an example of what such a frame class looks like here: https://devblogs.microsoft.com/premier-developer/dissecting-the-async-methods-in-c/?utm_source=pocket_mylist#the-state-machine