Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove IncludeToken as prep for caching parsing #3858

Merged
merged 62 commits into from
May 8, 2023

Conversation

keyboardDrummer
Copy link
Member

@keyboardDrummer keyboardDrummer commented Apr 10, 2023

Currently a file that's included is parsed differently. Its tokens are wrapped in an IncludeToken, that includes a reference to the including file.

To allow reusing parsed files in a cache, how the file is parsed should be independent of what triggered it to be parsed. To achieve that, this PR removes IncludeToken. To enable still determining whether a particular token was part of an include or not, tokens now store a Uri that uniquely identifies the document that token came from. The Uris of the root documents are stored in DefaultModuleDefinition, which is just below Program. Any token with a Uri that is not part of the root set is considered to be included.

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license.

@keyboardDrummer keyboardDrummer enabled auto-merge (squash) May 4, 2023 13:02
Copy link
Member

@MikaelMayer MikaelMayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an excellent start ! I think I have a few comments to address or answer but shouldn't be very long till merge.

Source/DafnyCore/AST/IncludeHandler.cs Show resolved Hide resolved
Source/DafnyCore/AST/Tokens.cs Show resolved Hide resolved
Source/DafnyCore/AST/Tokens.cs Outdated Show resolved Hide resolved
Source/DafnyCore/Coco/Scanner.frame Show resolved Hide resolved
Source/DafnyCore/DafnyConsolePrinter.cs Show resolved Hide resolved
Source/DafnyPipeline.Test/Issue1355.cs Show resolved Hide resolved
Source/DafnyPipeline.Test/PluginsTest.cs Show resolved Hide resolved
Source/DafnyPipeline.Test/Trivia.cs Show resolved Hide resolved
Copy link
Member

@MikaelMayer MikaelMayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting closer! A few more items

Source/DafnyTestGeneration/Utils.cs Outdated Show resolved Hide resolved
Test/git-issues/git-issue-3294.dfy.expect Outdated Show resolved Hide resolved
Source/DafnyCore/DafnyMain.cs Outdated Show resolved Hide resolved
Source/DafnyCore/Generic/Reporting.cs Outdated Show resolved Hide resolved
Copy link
Member

@MikaelMayer MikaelMayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job and perseverance going through this refactoring!

@keyboardDrummer keyboardDrummer merged commit eb45d23 into dafny-lang:master May 8, 2023
@keyboardDrummer keyboardDrummer deleted the removeIncludeToken branch May 9, 2023 07:44
fabiomadge added a commit to fabiomadge/dafny that referenced this pull request May 9, 2023
fabiomadge added a commit that referenced this pull request May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants