Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup "zola check" command by reusing the Client #2171

Merged
merged 2 commits into from
Apr 6, 2023

Conversation

MTRNord
Copy link
Contributor

@MTRNord MTRNord commented Apr 6, 2023

IMPORTANT: Please do not create a Pull Request adding a new feature without discussing it first.

The place to discuss new features is the forum: https://zola.discourse.group/
If you want to add a new feature, please open a thread there first in the feature requests section.

Sanity check:

  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Code changes

The code changes that zola check reuses the reqwest Client instead of creating it for every single request.
The benefit of this is a major speedup since it is able to reuse the connections. This is especially helpful for big pages like the one I am working on (17733 external links). The current stable way takes multiple hours, while this PR makes it go down to way less (as of writing it is still running but the progress bar I added in my local code predicts it to take 30-40 minutes instead of 14h+).

  • Are you doing the PR on the next branch?

If the change is a new feature or adding to/changing an existing one:

  • Have you created/updated the relevant documentation page(s)?

@MTRNord MTRNord changed the base branch from master to next April 6, 2023 19:51
@MTRNord MTRNord changed the title Mtr nord/speedup check Speedup "zola check" command by reusing the Client Apr 6, 2023
@Keats Keats merged commit 1ed722c into getzola:next Apr 6, 2023
@Keats
Copy link
Collaborator

Keats commented Apr 6, 2023

I just realised, the commit says it adds timeout but the code doesn't?

@MTRNord
Copy link
Contributor Author

MTRNord commented Apr 6, 2023

I just realised, the commit says it adds timeout but the code doesn't?

Ooops that's 100% on me. I removed that one after I opened the PR after realizing reqwest actually already defaults to 30s these days (It didn't in the past. So I naturally just gravitated to add it, but while checking the Docs I saw they default to it now. The commit is wrong because I used amend... Sorry!)

@MTRNord MTRNord deleted the MTRNord/speedup-check branch April 11, 2023 11:40
peterprototypes pushed a commit to peterprototypes/zola that referenced this pull request Sep 12, 2023
* Reuse Client when checking urls and add timeout for requests
technimad pushed a commit to technimad/zola that referenced this pull request Sep 30, 2023
* Reuse Client when checking urls and add timeout for requests
Keats pushed a commit that referenced this pull request Dec 18, 2023
* Reuse Client when checking urls and add timeout for requests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants