Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grafana: 10.3.3 -> 10.4.0, explicitly declare subPackages, halve build time, reduce closure size by ~11.5% #292997

Merged
merged 3 commits into from
Mar 16, 2024

Conversation

Ma27
Copy link
Member

@Ma27 Ma27 commented Mar 3, 2024

Description of changes

Closes #294020.

The build itself is pretty quick now:

buildPhase completed in 2 minutes 46 seconds

in contrast to

buildPhase completed in 5 minutes 22 seconds

before on the same machine (Hetzner AX41 NVMe with AMD Ryzen 5 3600 6-Core Processor and 64GiB RAM).

Downside of that is that no tests are now executed, but I'm inclined to make that sacrifice for now. Especially considering that a fix for that is on the horizon with #284568.

Note: a draft until we have agreed on whether it's OK to ignore the tests (and whether we should proactively remove the test skip code-paths or wait for #284568).
Grafana from this branch works fine and nobody has stepped up so far.


Tested on my personal instance and I haven't spotted an issue so far.

This also decreases the runtime closure size from 419.1M to 371.0M (~11.5% reduction).
I analyzed this with diffoscope. The only file that changed is bin/grafana:

│ │   --- result/bin/grafana
│ ├── +++ result-old/bin/grafana
│ │ ├── readelf --wide --notes {}
│ │ │ @@ -5,8 +5,8 @@
│ │ │  
│ │ │  Displaying notes found in: .note.ABI-tag
│ │ │    Owner                Data size 	Description
│ │ │    GNU                  0x00000010	NT_GNU_ABI_TAG (ABI version tag)	   OS: Linux, ABI: 3.10.0
│ │ │  
│ │ │  Displaying notes found in: .note.go.buildid
│ │ │    Owner                Data size 	Description
│ │ │ +  Go                   0x00000053	GO BUILDID	  description data: 6a 79 49 37 78 39 63 72 51 52 5f 6d 39 45 6a 51 39 6a 35 45 2f 34 38 65 35 4e 4f 69 57 49 58 6d 74 6d 6b 4d 38 69 64 6a 71 2f 42 69 73 64 78 46 43 46 52 56 6c 66 69 55 63 4b 66 71 37 63 2f 72 47 32 6c 41 33 5f 5a 63 6b 58 38 57 5a 66 54 37 76 65 69 
│ │ │ -  Go                   0x00000053	GO BUILDID	  description data: 6a 79 49 37 78 39 63 72 51 52 5f 6d 39 45 6a 51 39 6a 35 45 2f 34 38 65 35 4e 4f 69 57 49 58 6d 74 6d 6b 4d 38 69 64 6a 71 2f 42 69 73 64 78 46 43 46 52 56 6c 66 69 55 63 4b 66 71 37 63 2f 71 36 38 64 48 79 61 38 4b 32 72 39 70 78 49 77 6b 30 4f 46 
│ │ ├── strings --all --bytes=8 {}
│ │ │ @@ -1,9 +1,9 @@
│ │ │  /nix/store/xmprbk52mlcdsljz66m8yf7cf0xf36n1-glibc-2.38-44/lib/ld-linux-x86-64.so.2
│ │ │ +jyI7x9crQR_m9EjQ9j5E/48e5NOiWIXmtmkM8idjq/BisdxFCFRVlfiUcKfq7c/rG2lA3_ZckX8WZfT7vei
│ │ │ -jyI7x9crQR_m9EjQ9j5E/48e5NOiWIXmtmkM8idjq/BisdxFCFRVlfiUcKfq7c/q68dHya8K2r9pxIwk0OF
│ │ │  __gmon_start__
│ │ │  ftruncate64
│ │ │  getpwuid_r
│ │ │  setreuid
│ │ │  pthread_mutex_lock
│ │ │  localtime
│ │ │  pthread_detach

To me this seems like a non-function difference.

Also, the executables bin/cmd, bin/kinds, bin/openapi3, bin/standalone are gone now.

  • bin/cmd seems like a testing tool for a token generator: https://github.com/grafana/grafana/blob/869b89dce45090cd982414479a7e7b3bd7953253/pkg/components/satokengen/cmd/main.go#L33, so should be OK to drop.
  • bin/openapi3 is located in scripts/ and referenced in upstream's Makefile. I don't expect it to be used in production.
  • bin/standalone can be one of pkg/tsdb/azuremonitor/standalone/, pkg/tsdb/grafana-testdata-datasource/standalone/ or pkg/tsdb/parca/standalone/ depending on which is selected first by getGoDirs. This seems like a testing thing anyways, but was broken already.
  • The only kinds/*.go with a func main is kinds/gen.go which has a //go:build ignore at the top, so it doesn't seem needed as well.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Ma27 Ma27 changed the title grafana: explicitly declare subPackages, half build time, reduce closure size by ~11.5% grafana: explicitly declare subPackages, halve build time, reduce closure size by ~11.5% Mar 3, 2024
@Ma27
Copy link
Member Author

Ma27 commented Mar 15, 2024

OK screw it. The tests fail again on 10.4 and I don't know why and these unbearable slow build are annoying. Grafana appears to work fine when built from branch, so I'll do the 10.4 update here.

If we want tests back, #284568 may help.

@Ma27 Ma27 marked this pull request as ready for review March 15, 2024 15:22
@Ma27 Ma27 changed the title grafana: explicitly declare subPackages, halve build time, reduce closure size by ~11.5% grafana: 10.3.3 -> 10.4.0, explicitly declare subPackages, halve build time, reduce closure size by ~11.5% Mar 15, 2024
The build itself is pretty quick now:

    buildPhase completed in 2 minutes 46 seconds

in contrast to

    buildPhase completed in 5 minutes 22 seconds

before on the same machine (Hetzner AX41 NVMe with
AMD Ryzen 5 3600 6-Core Processor and 64GiB RAM).

Downside of that is that no tests are now executed, but I'm inclined to
make that sacrifice for now. Especially considering that a fix for that
is on the horizon with NixOS#284568.
We're currently not executing any tests by specifying `subPackages`
explicitly. If we go back to doing that, this can be reverted, but for
now it's just dead code.
@K900
Copy link
Contributor

K900 commented Mar 16, 2024

Build and run-tested on aarch64-linux, SGTM.

@Ma27 Ma27 merged commit b08f192 into NixOS:master Mar 16, 2024
22 of 23 checks passed
@Ma27 Ma27 deleted the grafana-subpackages branch March 16, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants