Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: touch build.rs files #10092

Closed
wants to merge 1 commit into from
Closed

ci: touch build.rs files #10092

wants to merge 1 commit into from

Conversation

kt3k
Copy link
Member

@kt3k kt3k commented Apr 9, 2021

The current CI settings seem having trouble with branches which only include .js & .ts changes. If a branch only includes changes of .js and .ts it doesn't trigger rebuild of anything. (example: https://github.com/denoland/deno/runs/2304805006

This workaround updates the modified times of build.rs files and force rebuild the dependencies which are created from build.rs scripts.

@kt3k
Copy link
Member Author

kt3k commented Apr 9, 2021

release builds are now unexpectedly slow (like ~30 min) 😕 though debug build is quite fast (2 min)

the debug build only rebuilds deno_runtime and deno (this is what I expected to happen), but the release builds seem building almost everything. The effect of touching build.rs looks different in release and debug.

@piscisaureus
Copy link
Member

The current CI settings seem having trouble with branches which only include .js & .ts changes.

Touching build.rs works in the interim. The "right" solution is to make build.rs tell cargo what its dependencies are. It can do this by writing lines like cargo:rerun-if-changed=foo/bar.ts to stdout.

release builds are now unexpectedly slow (like ~30 min) 😕 though debug build is quite fast (2 min)

I wonder if this was just a coincidence. This happens locally:

piscisaureus@spirit:~/deno$ touch runtime/build.rs
piscisaureus@spirit:~/deno$ cargo build
   Compiling deno_runtime v0.10.1 (/home/piscisaureus/deno/runtime)
   Compiling deno v1.8.3 (/home/piscisaureus/deno/cli)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 37s
piscisaureus@spirit:~/deno$ cargo build --release
   Compiling deno_runtime v0.10.1 (/home/piscisaureus/deno/runtime)
   Compiling deno v1.8.3 (/home/piscisaureus/deno/cli)
    Finished release [optimized] target(s) in 2m 45s

@ry
Copy link
Member

ry commented Apr 9, 2021

it can do this by writing lines like cargo:rerun-if-changed=foo/bar.ts to stdout

We do this properly AFAIK

@piscisaureus
Copy link
Member

piscisaureus commented Apr 9, 2021

We do this properly AFAIK

Looking at target/debug/deno.d, it looks like we do indeed.

In the github actions run referenced by @kt3k, only one .js file (plus a test) was modified (03ddc1f). If I replicate that locally it works correctly:

piscisaureus@spirit:~/deno$ touch runtime/js/40_http.js
piscisaureus@spirit:~/deno$ cargo build
   Compiling deno_runtime v0.10.1 (/home/piscisaureus/deno/runtime)
   Compiling deno v1.8.3 (/home/piscisaureus/deno/cli)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 26s

Maybe what happened was that some time had passed between the commit that modified 40_http.js and the time that the CI build was triggered. If the github actions cache that got restored had files in it with mtimes later than the commit date, cargo would assume that 40_http.js was not modified since the last build.

This issue is not specific to .ts/js files though, it could also happen with .rs files.

Using commit timestamps to restore mtimes might not be ideal. If we ported this logic to GHA that would solve the problem.

@piscisaureus
Copy link
Member

If we ported this logic to GHA that would solve the problem.

FYI I've started writing a github action.

@kt3k
Copy link
Member Author

kt3k commented Apr 10, 2021

Maybe what happened was that some time had passed between the commit that modified 40_http.js and the time that the CI build was triggered. If the github actions cache that got restored had files in it with mtimes later than the commit date, cargo would assume that 40_http.js was not modified since the last build.

I agree with this reasoning about the issue. And if this is true, I think touching diff files between main and PR branch also could be a solution, i.e. something like git diff --name-only main HEAD | xargs touch (which should cause rebuild of PR's change) I'll do some experiment to see if it works or not (maybe tomorrow

@kt3k kt3k closed this Apr 11, 2021
@kt3k
Copy link
Member Author

kt3k commented Apr 11, 2021

Tried 'touch diff from main' workaround in crowl's prompt fallback branch #9376

In that branch false negative rebuilding happened in https://github.com/denoland/deno/runs/2314683208, but the build was fixed with the workaround https://github.com/denoland/deno/runs/2316199508, and it took reasonable amount of rebuilding time. (20 min in windows release build, a half compared with full build.

@kt3k kt3k deleted the ci/touch-build-rs branch April 11, 2021 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants