Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

💫 Tidy up dependencies #2152

Merged
merged 8 commits into from
Mar 29, 2018
Merged

💫 Tidy up dependencies #2152

merged 8 commits into from
Mar 29, 2018

Conversation

ines
Copy link
Member

@ines ines commented Mar 28, 2018

Description

We're always looking to drop unnecessary dependencies. While spaCy's dependencies are generally quite healthy (and mostly consist of our own libraries), we were still able to identify several libraries that we didn't actually need – and which dragged in a range of other dependencies.

We've already removed six (6d2c85f), html5lib (mostly a workaround for a six issue) and ftfy (ce6071c, ac88c72).

The PR also replaces requests with urllib / urllib2. After inspecting the code again, it turned out that the library was only used in two places in the CLI and only to fetch JSON files. The new solution works fine on Python 3 – the Python 2 compatibility workaround still needs to be tested in detail.

We'd also like to upgrade msgpack-python to the renamed msgpack (see here for details). The way the library was renamed was kind of awkward, and a recent update caused various problems for spaCy users. So we want to be careful here. Another difficulty is that this change needs to happen in sync with Thinc, which also depends on msgpack-python. Otherwise, it will result in clashing versions again.

One option to solve this could be to only require msgpack via thinc. This way, spaCy won't have to specify a separate dependency, and we won't have to worry about both msgpacks being out of sync.

Types of change

enhancement

Checklist

  • I have submitted the spaCy Contributor Agreement.
  • I ran the tests, and all new and existing tests passed.
  • My changes don't require a change to the documentation, or if they do, I've added all required information.

@ines ines added install Installation issues ⚠️ wip Work in progress labels Mar 28, 2018
@ines ines removed the ⚠️ wip Work in progress label Mar 29, 2018
@ines ines merged commit a609a1c into master Mar 29, 2018
@ines ines deleted the feature/tidy-up-dependencies branch March 29, 2018 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
install Installation issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants