Skip to content
This repository has been archived by the owner on Feb 1, 2023. It is now read-only.

Speed-ups (shasum + poetry install caching) #91

Merged
merged 8 commits into from
Mar 3, 2021
Merged

Speed-ups (shasum + poetry install caching) #91

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7aab14e
Add command to lazy get the sha1 command
skovhus Mar 3, 2021
ecf803f
Remove duplicated function
skovhus Mar 3, 2021
7343ddc
Simplify yarn command detection
skovhus Mar 3, 2021
297f51c
Speed up poetry install
skovhus Mar 3, 2021
861ab16
Use the fastest shasum available (30% speedup)
skovhus Mar 3, 2021
50906a8
Log error if hashing takes more than 3 seconds
skovhus Mar 3, 2021
d67bb8d
Disable pylint warning (we like globals)
skovhus Mar 3, 2021
477454f
Update docstring
skovhus Mar 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Speed up poetry install 
Two timings when updating version number in libs/api:

Before poetry cache: 59/57 seconds
After  poetry cache: 31/30 seconds
  • Loading branch information
@skovhus
skovhus committed Mar 3, 2021
commit 297f51c1c7419a137c2025d143f581c9a9766a47
9 changes: 9 additions & 0 deletions brick/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,17 @@
"node:12.13.1": "v6",
}

POETRY_CACHE_LOCATION = "/root/.cache/pypoetry"


def is_yarn_install_command(cmd):
return cmd.startswith("yarn") or cmd.startswith("yarn install")


def is_poetry_install_command(cmd):
return cmd.startswith("poetry install")


timings = []
cyan = "\x1b[36;21m"
green = "\x1b[32;21m"
Expand Down Expand Up @@ -143,6 +149,9 @@ def generate_run_command(cmd, run_flags):
location = f"{YARN_CACHE_LOCATION}/{cache_version}"
logger.debug(f"Using yarn cache located at {location}")
run_flags += [f"--mount=type=cache,target={location}"]
if is_poetry_install_command(cmd):
logger.debug(f"Using poetry cache located at {POETRY_CACHE_LOCATION}")
run_flags += [f"--mount=type=cache,target={POETRY_CACHE_LOCATION}"]
if (secrets or {}).items():
# Wrap the run command with a tar command
# to untar and cleanup after us
Expand Down