-
Notifications
You must be signed in to change notification settings - Fork 17.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/tools: automate major version upgrades for modules #32014
Comments
I think this would be really valuable to have. Semantic import versioning is a common frustration people have with modules, and anything we could do to make it a better user experience would be helpful. I'd hesitate to add this functionality to cmd/go. It has a lot of responsibilities, but it's focused on build and dependency management, and it would be good not to expand the scope too far beyond that. That said, there is some precedence in We should also consider how this fits into any long-term plans for automatic refactoring. For example, is this something gopls could do? Also, should we provide a way to migrate automatically across incompatible API changes? |
I definitely think it would be useful to provide a tool that can add (or update) the I'm less convinced on updating consumers; I think an author-side tool like |
I think this is important on the module consumer side, too, but I admit I don't have a sense of how frequent and how inconvenient these upgrades are. I spoke with @ianthehat this morning. There's some ongoing work adding automated refactoring through gopls. Some of the points below are being actively worked on, others are just ideas.
The last idea was upgrading a particular module, not replacing it with a new major version. But maybe we could work this in, too. |
I think it is (arguably) even more important to support the consumer side, or at least I wouldn't suggest forgoing the consumer side. It is not just how frequently the upgrades occur -- the adoption of modules by a v2+ dependency means a consumer needs to go through this process, even if there is no actual major version upgrade involved (or if the "major version" upgrade was solely due to the adoption of modules). As has been said before, one would hope there are usually more consumers than authors for most open source modules. I think an author has more moving parts and it is harder for an author to get right, but the consumer pain here is more frequent, I think. People are currently doing this with things like For me, the best answer is "the tooling handles it for you".
|
Hi @bcmills @jayconrod, I'm curious for any latest thinking here regarding a possible location for such a tool? Two possible locations discussed previously:
(And being part of cmd/go is not very desirable, I think). One way to look at it -- Go 1.4 introduced optional custom import path checking, such as:
In part to help smooth adoption and minimize disruption, https://godoc.org/golang.org/x/tools/cmd/fiximports
It's probably fair to say that Semantic Import Versioning is a larger change to the ecosystem than optional import path checking in Go 1.4... There's been a massive amount of effort that has gone into modules, of course. Mainly just wanted to circle back to this given there is likely to be another wave of new adoptors with Go 1.14. @marwan-at-work I don't want to put you on the spot, but would you be willing for https://github.com/marwan-at-work/mod to be used (by someone?) as a starting point for a standalone golang.org/x/tools tool, if that ends up being a desirable choice? Also, regarding the current dependencies for github.com/marwan-at-work/mod: Now that https://golang.org/x/mod exists, do would it be reasonable for those dependencies to be more stripped down to be friendly to living in golang.org/x/tools (again, if that ends up being desirable)? |
@thepudds, note that a key difference between semantic import versioning and custom import path checking is that unversioned import paths remain usable and supported for major versions 0 and 1 and repositories that lack explicit For folks who want to resume support of a previously-unversioned module path, we changed the So, while I agree that an automatic upgrade tool might be useful, I don't think that Go 1.14 makes it particularly more urgent than before. |
Hi @bcmills, for new adoptors, partly I am just thinking about how each major release since 1.11 has triggered at least a mini-wave of adoptors, but in addition Go 1.14 has this statement:
I was not trying to suggest that something about Go 1.14 itself makes this more urgent, but rather the (likely) uptick around Go 1.14 is another time to think about whether or not it makes sense to have a standalone tool in golang.org/x (or gopls) to help smooth the transition. Semantic Import Versioning as you know sometimes generates "interesting" discussions within pockets of the ecosytem (e.g., two semi-random samples: 1, 2). Personally, I would be happier knowing that a tool (or gopls) is at least planned to help take some of the friction out... |
We're currently looking to bring While this seems simple the user experience is far from a good one as the imports for all consumers need to updated from We've had a PR hanging for some time and I was looking to pull the trigger until I tested the flow on a separate repo which clearly demonstrated how painful this was going to be for existing users. Based on this experience I agree that a core tool automate this process is key to help eliminate developer pain when it comes to this area of |
@stevenh, note that as of Go 1.14 you can resume the |
ooo thanks for the heads up on that, worth considering. |
As a relative newb Go developer, I'm confused as heck by the module versioning process - thought I'd add my two pennies here. Does it need to be so hard to update to the latest major version? I understand the idea of the promise of backward compatibility at the minor version level, but can we not have the "default" mode of "go get -u update" to latest minor version safely, and let "go get -u=major" perhaps update major versions and happily break things? |
Here's another tool which does the same thing: https://github.com/icholy/gomajor |
I have also written a tool that does this: https://github.com/nathanjcochran/upgrade |
This is a spin out of #31543 ("cmd/go: creating v2+ modules has lower success rate than it could").
Background
Semantic import versioning places the major version in module paths and import paths for v2+ modules, such as:
module github.com/some/mod/v2
in the author'sgo.mod
.require github.com/some/mod/v2 v2.0.0
in the consumer'sgo.mod
.import "github.com/some/mod/v2/some/pkg"
in the consumer's.go
files, and in the author's.go
files when the module's packages import other packages within the same v2+ module.This approach has value, but empirically it currently seems it can be a challenge to do correctly the first time (e.g., modules with v2 major version semver tags that are missing the required
/v2
in their ownmodule
statements, or modules that accidentally do not update all import statements and accidentally depend on the v1 version of themselves, etc.)It also creates additional work for authors and consumers if a module's major version increments, such as from v1 to v2, or v3 to v4, etc.
Suggestion
Tooling should be able to help here in a substantial way.
github.com/marwan-at-work/mod is great for people who know about it and who are willing to trust it.
However, a tool from the broader community won't have the penetration and impact of something from golang.org, or at least it would likely take much longer to get a similar type of penetration following a typical trajectory for a community tool.
Therefore, the suggestion here is to create a golang.org/x utility that can edit
go.mod
and.go
files to simplify the workflow for authors and consumers of v2+ modules. It might be possible to portmarwan-at-work/mod
itself, especially with the creation in #31761 of the x/mod repo that exposes APIs for module mechanics (such as an API forgo.mod
parsing).Three sample use cases:
/vN
in the module path in themodule
statement in thego.mod
.go
files if needed/vN
in the module path in themodule
statement in thego.mod
.go
files if neededrequire
statement properly in the consumer'sgo.mod
.go
files if neededI think
marwan-at-work/mod
can currently do all of those things.Perhaps one day similar functionality could live in cmd/go (e.g., the closed #27248), but it seems more practical to start with something in golang.org/x.
Non-goals
The suggestion here is that this golang.org/x utility would not do anything with VCS tagging, nor do anything for creating
/vN
subdirectories for anyone following the "Major Subdirectory" approach. Also, it would probably be reasonable for a first version to error out if used by a consumer that has areplace
for the module of interest (or, that could be handled more gracefully).CC @bcmills @jayconrod @marwan-at-work
The text was updated successfully, but these errors were encountered: