x/build/internal/releasetargets: evaluate for each upcoming major Go release #40561

dmitshur · 2020-08-04T03:36:08Z

At the start of each code freeze, we should evaluate the choice of releaselets (builders that are used for constructing a release artifact) used to make the upcoming major Go release, and confirm they are the most appropriate choice out of what is available.

The start of a code freeze is a safe time to introduce a builder change, as it will be tested during pre-release versions, starting with beta 1 and onwards.

Making this a release blocker for Go 1.16 (and not okay after beta 1—it needs to be done before). Once completed, this issue should be moved to the next major release milestone.

Edit: Starting with Go 1.21, releaselets are only used for running tests. Release artifact construction happens by reproducibly cross-compiling a distpack distribution on a fixed builder.

CC @golang/release, @ianlancetaylor.

toothrot · 2020-11-05T19:16:06Z

/cc @cagedmantis

dmitshur · 2020-11-21T01:18:21Z

The @golang/release team has considered the current builders, and we are proposing the following builder changes for the Go 1.16 release:

go1.16*.linux-amd64.tar.gz

	Builder Name	Host Type
Current	linux-amd64-jessie	host-linux-jessie
New	linux-amd64	host-linux-stretch

(Or rather a new builder, identical to the linux-amd64 one, just named linux-amd64-stretch instead.)

Rationale: Debian 8 "Jessie" Long Term Support (LTS) has ended on June 30, 2020 (see https://wiki.debian.org/LTS). We want to start using the next stable release, Debian 9 "Stretch", which has LTS support until June 2022.

Note that this has caused #31293 in April 2019 when this change was made unintentionally as part of a minor release, and it was reverted. We hope that it should be okay to try again for Go 1.16 a year later, since this is being done intentionally this time, and for a major release. CC @ianlancetaylor, @jayconrod, @bradfitz.

go1.16*.linux-386.tar.gz

	Builder Name	Host Type
Current	linux-386	host-linux-jessie
New	linux-386-stretch	host-linux-stretch

Rationale: Same as the one above for linux/amd64.

go1.16*.linux-armv6l.tar.gz

	Builder Name	Host Type
Current	linux-arm	host-linux-arm-scaleway
New	linux-arm-aws	host-linux-arm-aws

Rationale: The replacement builder has proven to be more reliable, and it is easier for us to maintain.

go1.16*.linux-arm64.tar.gz

	Builder Name	Host Type
Current	linux-arm64-packet	host-linux-arm64-packet
New	linux-arm64-aws	host-linux-arm64-aws

Rationale: The replacement builder has proven to be more reliable, and it is easier for us to maintain.

Also see #42304 (comment).

Feedback is welcome: we are happy to alter the plan based on information we haven't already considered. These builder changes are planned for Go 1.16 Beta 1, and we will re-evaluate this as needed based on any new findings and feedback.

gopherbot · 2020-12-08T04:08:51Z

Change https://golang.org/cl/276034 mentions this issue: cmd/release, dashboard: implement builder plan for Go 1.16

See https://golang.org/issues/40561#issuecomment-731482962 for the plan description and rationale. Add new builder definitions that are needed for the plan. Don't rely on generic builder name like "linux-amd64" to avoid a repeat of golang.org/issue/31293; use a specific builder name instead. Remove support and test cases for Go 1.13, it's unsupported per release policy. For golang/go#40561. Change-Id: I070744e15be9f1932d649a27ee7e4599e467553f Reviewed-on: https://go-review.googlesource.com/c/build/+/276034 Trust: Dmitri Shuralyov <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Carlos Amedee <[email protected]>

dmitshur · 2020-12-09T05:08:19Z

CL 276034 has implemented the aforementioned plan for Go 1.16.

We'll need to revisit this issue next time during the Go 1.17 development cycle. (If we discover problems with the Go 1.16 plan, we'll come back here, or file new issues.) Moving to the next milestone for now.

thanm · 2020-12-15T14:59:49Z

While working on issue #39326, it has become clear to me that the C compile we have installed on our windows gomotes (at least the ones I have used) is pretty ancient -- GCC 5. It would be great if we could update the version to something a bit more recent (V10 maybe). Also worth noting that many folks are now using clang instead of gcc on windows -- it would be great if our gomotes had clang available as well.

dmitshur · 2020-12-15T15:30:14Z

@thanm Can you please file a separate issue to track that work, and CC @golang/release? It may need more information/investigation. Thanks.

thanm · 2020-12-15T16:13:04Z

Filed #43195. Thanks.

gopherbot · 2020-12-15T17:22:29Z

Change https://golang.org/cl/278357 mentions this issue: cmd/release: update checkRelocations for Go 1.16

The plan in https://golang.org/issue/40561#issuecomment-731482962 involved updating the linux-amd64 target from Debian Jessie to Debian Stretch, which in turn comes with a newer version of GCC. checkRelocations was added in CL 171317 in response to a bad minor release that was accidentally issued without the intended fix, and needs to be updated for Go 1.16 and newer releases. It's not clear if we want to maintain it indefinitely for future Go versions, but keep it for a bit longer. Add a note to make it clear that it can be removed as needed in the future. For golang/go#40561. Updates golang/go#31293. Change-Id: I2da419eff6379575eb2648787f0dac6bba07b304 Reviewed-on: https://go-review.googlesource.com/c/build/+/278357 Trust: Dmitri Shuralyov <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Alexander Rakoczy <[email protected]> Reviewed-by: Carlos Amedee <[email protected]>

gopherbot · 2021-04-23T17:00:26Z

Change https://golang.org/cl/313070 mentions this issue: dashboard: make FreeBSD 11.4 and 12.2 builders the default

With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <[email protected]> Run-TryBot: Carlos Amedee <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

cagedmantis · 2021-04-30T22:02:20Z

The @golang/release team has considered the current builders, and we are proposing the following builder change for the Go 1.17 release:

go1.17.*.freebsd-amd64.tar.gz

	Builder Name	Host Type
Current	freebsd-amd64-11_2	host-freebsd-11_2
New	freebsd-amd64-11_4	host-freebsd-11_4

Rationale: Using the latest "patch" release of the oldest supported "major" release of FreeBSD.

#45727 (comment)

go1.17.*.freebsd-386.tar.gz

	Builder Name	Host Type
Current	freebsd-386-11_2	host-freebsd-11_2
New	freebsd-386-11_4	host-freebsd-11_4

Rationale: Using the latest "patch" release of the oldest supported "major" release of FreeBSD.

#45727 (comment)

gopherbot · 2021-04-30T22:03:30Z

Change https://golang.org/cl/315889 mentions this issue: cmd/release: start using the FreeBSD 11.4 builder for Go 1.17

There won't be new Go 1.18 releases per go.dev/doc/devel/release#policy. For golang/go#40561. Change-Id: Id9c34368e97143198419fdbf75c8fcfbf853e6ab Reviewed-on: https://go-review.googlesource.com/c/build/+/503755 Run-TryBot: Dmitri Shuralyov <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Heschi Kreinick <[email protected]> Reviewed-by: Carlos Amedee <[email protected]>

It's a no-op since the only newly added port is wasip1/wasm, and we won't be making binary releases for it at this time. For golang/go#40561. For golang/go#58141. Change-Id: I0c9932fdfca0842c6860bca0cdbc4b1d64fdefff Reviewed-on: https://go-review.googlesource.com/c/build/+/503756 Reviewed-by: Heschi Kreinick <[email protected]> Reviewed-by: Carlos Amedee <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> TryBot-Result: Gopher Robot <[email protected]>

A longtest builder for the darwin/amd64 port was added in the Go 1.21 development cycle. Start using it as for non-advisory release testing. For golang/go#40561. For golang/go#35678. For golang/go#29252. Change-Id: Ic65e84e5e10bcb786cb28c36c1d1b137e2f6202e Reviewed-on: https://go-review.googlesource.com/c/build/+/503758 Run-TryBot: Dmitri Shuralyov <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Heschi Kreinick <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]>

gopherbot · 2023-06-20T12:33:31Z

Change https://go.dev/cl/504525 mentions this issue: internal/releasetargets: set LongTestBuilder for linux-arm64

The linux/arm64 port is first class and a longtest builder was added for it during the Go 1.20 development cycle. It's a mandatory trybot on 1.20+ release branches and is working well. The builder recently found a real problem in go.dev/issue/60825, which has been resolved by now. Upgrade this builder from an advisory TryBot to have more complete release test coverage. For golang/go#40561. For golang/go#49649. For golang/go#29252. Change-Id: I1765e820deb0d34b7bb9a72aba397d87c1369ced Reviewed-on: https://go-review.googlesource.com/c/build/+/504525 Run-TryBot: Dmitri Shuralyov <[email protected]> Reviewed-by: Heschi Kreinick <[email protected]> Reviewed-by: Cherry Mui <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]>

gopherbot · 2023-09-08T18:17:09Z

Change https://go.dev/cl/527017 mentions this issue: internal/releasetargets: drop Go 1.19 targets

There won't be new Go 1.19 releases per go.dev/doc/devel/release#policy. For golang/go#62076. For golang/go#40561. Change-Id: I30e09c9f47ec0006caeb67e15e03b8cdbff7c175 Reviewed-on: https://go-review.googlesource.com/c/build/+/527017 Auto-Submit: Dmitri Shuralyov <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Carlos Amedee <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

gopherbot · 2023-12-06T17:01:45Z

Change https://go.dev/cl/547935 mentions this issue: internal/releasetargets: regenerate all ports for Go 1.22

It's the same list as 1.21, with the addition of the openbsd/ppc64 port. Generated with 'go generate'. For golang/go#40561. For golang/go#56001. Change-Id: I93d2cfd191134f524255393e8b60bf6ce7328940 Reviewed-on: https://go-review.googlesource.com/c/build/+/547935 Reviewed-by: Dmitri Shuralyov <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Carlos Amedee <[email protected]>

dmitshur · 2023-12-07T18:03:35Z

CL 547935 takes care of the port list for Go 1.22.

After Go 1.22.0 is out, all release artifacts we produce will be via reproducible cross-compilation and distpack, so the work of "evaluate the choice of releaselets (builders that are used for constructing a release artifact)" that this issue tracks becomes obsolete. We still need to handle the port list, though it'll be possible to replace its generation with a run-time computation if that's easier. And the releasetargets package can be simplified. We do still need to continuously maintain and update OS versions, but that's out of scope for here.

Dealing with all that is for after Go 1.22.0 is out, so moving this to the next milestone to revisit later.

gopherbot · 2024-02-14T21:21:52Z

Change https://go.dev/cl/564217 mentions this issue: internal/releasetargets: update MinMacOSVersion for Go 1.23

gopherbot · 2024-02-14T21:21:54Z

Change https://go.dev/cl/564216 mentions this issue: internal/releasetargets: drop Go 1.20 release targets

gopherbot · 2024-02-14T22:56:00Z

Change https://go.dev/cl/564255 mentions this issue: internal/releasetargets: delete fields not relevant to distpack releases

Delete the old before starting to update for the new. "Targets for release 1.21" and newer in releases.txt doesn't change. For golang/go#40561. Change-Id: I40abd20bbee165a641e2a22a6e791cfbdf950058 Reviewed-on: https://go-review.googlesource.com/c/build/+/564216 Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Than McIntosh <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

For golang/go#40561. Fixes golang/go#64207. Change-Id: I1bbd60d0c2bcae28856c7f7c8293cb2f5a2397f5 Reviewed-on: https://go-review.googlesource.com/c/build/+/564217 Reviewed-by: Than McIntosh <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]>

The Target struct fields Builder, LongTestBuilder, and BuildOnly were applicable to releases prior to distpacks (i.e., Go 1.20 and older). By now, all releases are built on a secured Linux machine (and verified by rebuilding on another secured Windows machine), installers are built by the same pipeline that's responsible for performing signing, and the tests are run via the "advisory builder" loop. These fields have become meaningless and confusing, so drop them. Print whether a port is primary or not in releases.txt, so that if a map entry like "linux-amd64": &Target{} that only has implicit effects left by now is accidentally removed from allReleases, it'll be easy to spot the effect via the releases.txt diff. There's a bit more to do before the entire package becomes obsolete. For golang/go#40561. Change-Id: Ib66ca6958db695db0093556edc822dcdfbfdde0d Reviewed-on: https://go-review.googlesource.com/c/build/+/564255 Reviewed-by: Dmitri Shuralyov <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Michael Knyszek <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]>

gopherbot · 2024-05-13T17:01:24Z

Change https://go.dev/cl/585216 mentions this issue: internal/releasetargets: add openbsd/ppc64 target for Go 1.22 onwards

The release target list has two underlying sources: the allReleases map and the allPorts map. The former is a small curated map, mostly used to indicate which ports are first-class, and to set a few port properties. The latter is generated from 'go tool dist list' output for each major Go release. Unfortunately, the logic in TargetsForGo1Point ended up not taking into account that CL 547935 added allports/go1.22.txt with the openbsd/ppc64 port in it because the allReleases map didn't have an entry for go1.22. As a result, we haven't been making releases for the openbsd/ppc64 port as intended. :( Fix this by making sortedReleases consult both sources. For golang/go#40561. For golang/go#56001. Change-Id: Icbabad6f7e2920c1b24a69e309d65aea77c9c461 Reviewed-on: https://go-review.googlesource.com/c/build/+/585216 LUCI-TryBot-Result: Go LUCI <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Cherry Mui <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

gopherbot · 2024-05-24T19:44:47Z

Change https://go.dev/cl/588137 mentions this issue: internal/releasetargets: regenerate all ports for Go 1.23

The release freeze for Go 1.23 has started, so now is a good time to regenerate this list. Done with 'go generate'. The list is the same as for Go 1.22 with the addition of the new openbsd/riscv64 port. For golang/go#40561. For golang/go#55999. Change-Id: Ib920c3b4337fcd9aa4a765e5622e6c158f829bda Reviewed-on: https://go-review.googlesource.com/c/build/+/588137 Reviewed-by: Carlos Amedee <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

dmitshur · 2024-05-24T20:44:10Z

Done for the Go 1.23 cycle.

dmitshur added Builders x/build issues (builders, bots, dashboards) release-blocker recurring Issues that should never be closed, but moved to the next milestone once fixed in the current one. labels Aug 4, 2020

dmitshur added this to the Go1.16 milestone Aug 4, 2020

dmitshur mentioned this issue Aug 4, 2020

x/build/cmd/release: update to freebsd-{amd64,386}-11_2 builder for 1.15 RC 2+ #40563

Closed

toothrot added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Aug 4, 2020

dmitshur mentioned this issue Oct 30, 2020

x/build/env/linux-arm64/packet: 8 Packet linux/arm64 machines missing (late-October 2020) #42304

Closed

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Nov 19, 2020

dmitshur self-assigned this Dec 3, 2020

dmitshur modified the milestones: Go1.16, Go1.17 Dec 9, 2020

dmitshur removed their assignment Dec 9, 2020

dmitshur changed the title ~~x/build/cmd/release: evaluate builders used for upcoming major Go release~~ x/build/cmd/release: evaluate releaselets used for each upcoming major Go release Dec 15, 2020

dmitshur mentioned this issue Dec 15, 2020

x/build/cmd/release: linux-armv6l release tests aren't passing #43202

Closed

dmitshur mentioned this issue Jan 29, 2021

x/build: document that Ubuntu 14.04 doesn't meet minimum requirement for Go 1.16 binary release #43996

Closed

dmitshur self-assigned this Nov 29, 2023

dmitshur modified the milestones: Go1.22, Go1.23 Dec 7, 2023

dmitshur removed their assignment Dec 7, 2023

dmitshur self-assigned this Apr 19, 2024

dmitshur added the FixPending Issues that have a fix which has not yet been reviewed or submitted. label May 24, 2024

dmitshur removed the FixPending Issues that have a fix which has not yet been reviewed or submitted. label May 24, 2024

dmitshur modified the milestones: Go1.23, Go1.24 May 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/build/internal/releasetargets: evaluate for each upcoming major Go release #40561

x/build/internal/releasetargets: evaluate for each upcoming major Go release #40561

dmitshur commented Aug 4, 2020 •

edited

Loading

toothrot commented Nov 5, 2020

dmitshur commented Nov 21, 2020 •

edited

Loading

gopherbot commented Dec 8, 2020

dmitshur commented Dec 9, 2020

thanm commented Dec 15, 2020

dmitshur commented Dec 15, 2020 •

edited

Loading

thanm commented Dec 15, 2020

gopherbot commented Dec 15, 2020

gopherbot commented Apr 23, 2021

cagedmantis commented Apr 30, 2021

gopherbot commented Apr 30, 2021

gopherbot commented Jun 20, 2023

gopherbot commented Sep 8, 2023

gopherbot commented Dec 6, 2023

dmitshur commented Dec 7, 2023

gopherbot commented Feb 14, 2024

gopherbot commented Feb 14, 2024

gopherbot commented Feb 14, 2024

gopherbot commented May 13, 2024

gopherbot commented May 24, 2024

dmitshur commented May 24, 2024

x/build/internal/releasetargets: evaluate for each upcoming major Go release #40561

x/build/internal/releasetargets: evaluate for each upcoming major Go release #40561

Comments

dmitshur commented Aug 4, 2020 • edited Loading

toothrot commented Nov 5, 2020

dmitshur commented Nov 21, 2020 • edited Loading

go1.16*.linux-amd64.tar.gz

go1.16*.linux-386.tar.gz

go1.16*.linux-armv6l.tar.gz

go1.16*.linux-arm64.tar.gz

gopherbot commented Dec 8, 2020

dmitshur commented Dec 9, 2020

thanm commented Dec 15, 2020

dmitshur commented Dec 15, 2020 • edited Loading

thanm commented Dec 15, 2020

gopherbot commented Dec 15, 2020

gopherbot commented Apr 23, 2021

cagedmantis commented Apr 30, 2021

go1.17.*.freebsd-amd64.tar.gz

go1.17.*.freebsd-386.tar.gz

gopherbot commented Apr 30, 2021

gopherbot commented Jun 20, 2023

gopherbot commented Sep 8, 2023

gopherbot commented Dec 6, 2023

dmitshur commented Dec 7, 2023

gopherbot commented Feb 14, 2024

gopherbot commented Feb 14, 2024

gopherbot commented Feb 14, 2024

gopherbot commented May 13, 2024

gopherbot commented May 24, 2024

dmitshur commented May 24, 2024

dmitshur commented Aug 4, 2020 •

edited

Loading

dmitshur commented Nov 21, 2020 •

edited

Loading

dmitshur commented Dec 15, 2020 •

edited

Loading