Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Execution branch coverage report #4755

Merged
merged 17 commits into from
Nov 9, 2023
Merged

feat: Execution branch coverage report #4755

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
a70859f
Slightly hacky working first pass
robin-aws Nov 7, 2023
4bc558f
Rename existing —coverage-report to —verification-coverage-report
robin-aws Nov 8, 2023
b1254a4
Improve coverage report formatting, hook up to stdlibs, start to impr…
robin-aws Nov 8, 2023
fb55df8
Integration test mostly working
robin-aws Nov 8, 2023
99ff54b
Fix regression in report naming
robin-aws Nov 8, 2023
b1f5da6
Revert runtime change
robin-aws Nov 8, 2023
084f928
Nonuniform test
robin-aws Nov 8, 2023
ed93db2
Fix paths in expect file
robin-aws Nov 8, 2023
46d7331
Test for unsupported feature too
robin-aws Nov 8, 2023
6f2a5d4
Mention —coverage-report for dafny test
robin-aws Nov 8, 2023
b0b7fc5
Update features table
robin-aws Nov 8, 2023
e40be50
Improve encapsulation
robin-aws Nov 8, 2023
06a9068
Release note, typo, whitespace
robin-aws Nov 9, 2023
87aa068
Typo and library backend
robin-aws Nov 9, 2023
55616a1
Merge branch 'master' into runtime-coverage-report
robin-aws Nov 9, 2023
9262299
Fix feature support table
robin-aws Nov 9, 2023
64ac12d
Merge branch 'master' of https://github.com/dafny-lang/dafny into run…
robin-aws Nov 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rename existing —coverage-report to —verification-coverage-report
  • Loading branch information
@robin-aws
robin-aws committed Nov 8, 2023
commit 4bc558f219573ec970f56a79b765b8155c9bb7f0
4 changes: 2 additions & 2 deletions Source/DafnyCore/Options/CommonOptionBag.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,15 +214,15 @@ May slow down verification slightly.
May produce spurious warnings.") {
IsHidden = true
};
public static readonly Option<string> VerificationCoverageReport = new("--coverage-report",
public static readonly Option<string> VerificationCoverageReport = new("--verification-coverage-report",
"Emit verification coverage report to a given directory, in the same format as a test coverage report.") {
ArgumentHelpName = "directory"
};
public static readonly Option<bool> NoTimeStampForCoverageReport = new("--no-timestamp-for-coverage-report",
"Write coverage report directly to the specified folder instead of creating a timestamped subdirectory.") {
IsHidden = true
};
public static readonly Option<string> ExecutionCoverageReport = new("--test-coverage-report",
public static readonly Option<string> ExecutionCoverageReport = new("--coverage-report",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could call this --execution-coverage-report, since that term shows up in the variable name and the description text. :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did consider that, but then it would feel weird to not further qualify dafny generate-tests --coverage-report as well, perhaps as --expected-execution-coverage-report. Given the generate-tests one is already released I'm more inclined to leave both with the simpler name.

"Emit execution coverage report to a given directory.") {
ArgumentHelpName = "directory"
};
Expand Down
4 changes: 2 additions & 2 deletions docs/DafnyRef/UserGuide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1512,12 +1512,12 @@ included in the proof.
These options can be specified in `dfyconfig.toml`, and this is typically the most convenient way to use them with the IDE.

More detailed information is available using either the `--log-format
text` or `--coverage-report` option to `dafny verify`. The former will
text` or `--verification-coverage-report` option to `dafny verify`. The former will
include a list of proof dependencies (including source location and
description) alongside every assertion batch in the generated log
whenever one of the two warning options above is also included. The
latter will produce a highlighted HTML version of your source code, in
the same format used by `dafny generate-tests --coverage-report`,
the same format used by `dafny generate-tests --verification-coverage-report`,
indicating which parts of the program were used, not used, or partly
used in the verification of the entire program.

Expand Down
2 changes: 1 addition & 1 deletion docs/dev/news/4625.feat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
The new `--coverage-report` flag to `dafny verify` creates an HTML report highlighting which portions of the program were and were not necessary for verification. The format is the same as for `dafny generate-tests --coverage-report` and files from the two commands can be merged.
The new `--verification-coverage-report` flag to `dafny verify` creates an HTML report highlighting which portions of the program were and were not necessary for verification. The format is the same as for `dafny generate-tests --coverage-report` and files from the two commands can be merged.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for being thorough about where you change the name of this flag!