Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cut Windows CI build time in half to 15 min #586

Merged
merged 2 commits into from
Sep 5, 2020
Merged

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Sep 5, 2020

Description of proposed changes

Attempt at speeding up Windows builds by updating conda version. For Python 3.6, our current Windows build takes ~31min which is too slow!! Should at least half that time to ~15 minutes to make it more in line with macOS/Linux builds. The bulk of the time (~24min) is spent just installing dependencies using conda, so that will be what this PR will focus on optimizing.

Before

Windows (~31min) macOS (~11min)
Windows Python 3.6 CI in 31min macOS Python 3.6 CI in 11min

After

Windows (~15min) macOS (~9min)
Windows Python 3.6 CI in 15min macOS Python 3.6 CI in 9min

TODO:

  • Bump setup-miniconda from 1.6.0 to 1.7.0, also note name change from goanpeca/setup-miniconda to conda-incubator/setup-miniconda
  • Try using 'latest' miniconda version 4.8.3 instead of 4.6.14 on Windows
  • Cache dependencies?

Towards #584

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If adding new functionality, add an example to docstrings or tutorials.

@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Sep 5, 2020
This would update to miniconda version 4.8.3
, which is a pretty significant upgrade for the
Github Actions Windows runner still stuck at
4.6.14 (as of 6 September 2020).
@weiji14 weiji14 marked this pull request as ready for review September 5, 2020 23:34
with:
python-version: ${{ matrix.python-version }}
channels: conda-forge
miniconda-version: "latest"
Copy link
Member Author

@weiji14 weiji14 Sep 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll like this @seisman, The Windows conda install step speed up (~24min -> ~8min) comes from just adding this one line! What it does (on Windows) is to update the miniconda version from 4.6.14 to 4.8.3, and this would use a newer/faster conda solver, among other things.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also speed up macOS. Perhaps we should also update ci_tests_dev.yaml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really by much, I think we can do it another time since that’s not really a bottleneck.

Copy link
Member

@seisman seisman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@weiji14 weiji14 merged commit db42684 into master Sep 5, 2020
@weiji14 weiji14 deleted the setup-miniconda-1.7.0 branch September 5, 2020 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants