Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New resolver: Try == specifiers first to catch simple incompatibilities faster #9185

Closed
uranusjr opened this issue Nov 30, 2020 · 1 comment · Fixed by #9211
Closed

New resolver: Try == specifiers first to catch simple incompatibilities faster #9185

uranusjr opened this issue Nov 30, 2020 · 1 comment · Fixed by #9211
Milestone
20.3.2

Comments

@uranusjr
Copy link
Member

Idea I have while working on #9011 (comment)

This input:

tensorboard==2.2.2
torch==1.7.0
tqdm>=4.54

nltk==3.5.0
transformers==3.5.0
tokenizers==0.9.4
jsonlines>=1.2
sacrebleu>=1.4

pytest>=6.1
pytest-cov>=2.10
codecov>=2.1
flake8>=3.8
black==20.8b1

Leads to this eventual error:

The conflict is caused by:
    The user requested tokenizers==0.9.4
    transformers 3.5.0 depends on tokenizers==0.9.3

But the resolver needs a very long time to reach it, because… the packages started with to and tr are checked last, after the resolver successfully worked out all the other complex open specifier combinations 🤦

I think we can make things a bit better by checking for == (and ===) specifiers and do those packages first. This is similar to how we previously “optimised” the process by using len(candidates) (the less candidates a package has, the more likely it causes conflicts, so we check them first to catch simple cases early), but without needing to actually fetch the list of candidates eagerly.
@brainwane brainwane added this to the 20.3.1 milestone Nov 30, 2020
@pradyunsg
Copy link
Member

I like it. Let's do this. :)

@kdelee kdelee mentioned this issue Nov 30, 2020
Closed
@dstufft dstufft mentioned this issue Nov 30, 2020
Closed
@uranusjr uranusjr mentioned this issue Dec 3, 2020
Merged
@pfmoore pfmoore mentioned this issue Dec 3, 2020
Closed
bors bot referenced this issue in duckinator/emanate Dec 15, 2020
@bors @pyup-bot
Merge #204 #205
2595785 
204: Update pip to 20.3.3 r=duckinator a=pyup-bot


This PR updates [pip](https://pypi.org/project/pip) from **20.3.1** to **20.3.3**.



<details>
  <summary>Changelog</summary>
  
  
   ### 20.3.3
   ```
   ===================

Bug Fixes
---------

- Revert &quot;Skip candidate not providing valid metadata&quot;, as that caused pip to be overeager about downloading from the package index. (`9264 &lt;https://github.com/pypa/pip/issues/9264&gt;`_)
   ```
   
  
  
   ### 20.3.2
   ```
   ===================

Features
--------

- New resolver: Resolve direct and pinned (``==`` or ``===``) requirements first
  to improve resolver performance. (`9185 &lt;https://github.com/pypa/pip/issues/9185&gt;`_)
- Add a mechanism to delay resolving certain packages, and use it for setuptools. (`9249 &lt;https://github.com/pypa/pip/issues/9249&gt;`_)

Bug Fixes
---------

- New resolver: The &quot;Requirement already satisfied&quot; log is not printed only once
  for each package during resolution. (`9117 &lt;https://github.com/pypa/pip/issues/9117&gt;`_)
- Fix crash when logic for redacting authentication information from URLs
  in ``--help`` is given a list of strings, instead of a single string. (`9191 &lt;https://github.com/pypa/pip/issues/9191&gt;`_)
- New resolver: Correctly implement PEP 592. Do not return yanked versions from
  an index, unless the version range can only be satisfied by yanked candidates. (`9203 &lt;https://github.com/pypa/pip/issues/9203&gt;`_)
- New resolver: Make constraints also apply to package variants with extras, so
  the resolver correctly avoids backtracking on them. (`9232 &lt;https://github.com/pypa/pip/issues/9232&gt;`_)
- New resolver: Discard a candidate if it fails to provide metadata from source,
  or if the provided metadata is inconsistent, instead of quitting outright. (`9246 &lt;https://github.com/pypa/pip/issues/9246&gt;`_)

Vendored Libraries
------------------

- Update vendoring to 20.8

Improved Documentation
----------------------

- Update documentation to reflect that pip still uses legacy resolver by default in Python 2 environments. (`9269 &lt;https://github.com/pypa/pip/issues/9269&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pip
  - Changelog: https://pyup.io/changelogs/pip/
  - Homepage: https://pip.pypa.io/
</details>



205: Update pytest to 6.2.1 r=duckinator a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **6.1.2** to **6.2.1**.



*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
20.3.2
Development

Successfully merging a pull request may close this issue.

Resolve direct and pinned requirements first uranusjr/pip
3 participants
@uranusjr @brainwane @pradyunsg