Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance benchmarking #336

Merged
merged 34 commits into from
Sep 8, 2023
Merged

Performance benchmarking #336

merged 34 commits into from
Sep 8, 2023

Conversation

NicolasRouquette
Copy link
Collaborator

The API is backwards compatible but offers new APIs for compose and quotient that allow specifying the list of tactics to be used and provide a tuple result, the resulting contract, and the list of tactic usage information.

iincer and others added 27 commits August 19, 2023 09:01
…new member function to provide this capability
@codecov
Copy link

codecov bot commented Sep 5, 2023

Codecov Report

Merging #336 (6cb8444) into main (ae3b55d) will increase coverage by 2.15%.
Report is 2 commits behind head on main.
The diff coverage is 83.89%.

@@            Coverage Diff             @@
##             main     #336      +/-   ##
==========================================
+ Coverage   79.61%   81.77%   +2.15%     
==========================================
  Files          16       16              
  Lines        1982     2052      +70     
  Branches      462      440      -22     
==========================================
+ Hits         1578     1678     +100     
+ Misses        273      238      -35     
- Partials      131      136       +5     
Files Changed Coverage Δ
src/pacti/terms/polyhedra/serializer.py 79.37% <ø> (+6.25%) ⬆️
src/pacti/terms/polyhedra/polyhedra.py 82.09% <73.75%> (+3.30%) ⬆️
src/pacti/contracts/polyhedral_iocontract.py 82.00% <88.23%> (+0.39%) ⬆️
src/pacti/iocontract/iocontract.py 80.13% <97.77%> (+2.31%) ⬆️
src/pacti/iocontract/__init__.py 100.00% <100.00%> (ø)
src/pacti/terms/polyhedra/syntax/grammar.py 88.97% <100.00%> (+1.72%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@iincer
Copy link
Collaborator

iincer commented Sep 5, 2023

Hi @NicolasRouquette, the code quality checks are passing, which is great. The code coverage decreased as a result of the new functionality for performance. Would you be able to write a couple of tests to exercise the new functionality?
The report is https://github.com/pacti-org/pacti/pull/336/checks?check_run_id=16494187644

In addition to make test, we can run make coverage to generate an html page with coverage data.

A list of terms not containing any variables in `vars_to_elim`
and which, in the context provided, imply the terms contained in the
calling termlist.
A tuple consisting of:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check if mkdocs will pick this docstring correctly. Newlines need additional indentation for them to show up as part of the same structure. Since you have "-", I am not sure what will happen.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you generate the doc?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs are generated from the pacti-docs repo.
We can look at the documentation locally via the command make docs-serve

@NicolasRouquette
Copy link
Collaborator Author

Yes, I'll add some additional tests to improve the coverage.

@NicolasRouquette
Copy link
Collaborator Author

I managed to get the code coverage > 80%.

For the remaining, we must create a contrived PolyhedralTermList and call some methods like _transform with arguments to exercise the missing code branches.

Also, can you suggest pathological inequalities that could trip the untested branches in the various tactics?

@iincer iincer linked an issue Sep 7, 2023 that may be closed by this pull request
Copy link
Collaborator

@iincer iincer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending for another PR:

  • Improve the docstrings to make sure the documentation can process them
  • Use less type names for the tactic statistics inside IO contract

@iincer iincer merged commit 69697b7 into main Sep 8, 2023
3 checks passed
@iincer iincer deleted the performance-benchmarking branch September 8, 2023 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature request] Adding division operation to the parser
2 participants