Prelude additions

[djc]

Here's a first take on the plan to update the prelude for 2021 edition.

Rendered
March 3 feedback summary

[jyn514]

Adding a trait to the prelude can cause compatibility issues

This is true, but AFAIK T-libs doesn't actually consider that a breaking change (since otherwise every time they added an inherent method, or a new method to a trait, it could break code). Not sure if you want to mention that or not.

[scottmcm]

I think the official terminology is that it's a "minor change": https://rust-lang.github.io/rfcs/1105-api-evolution.html#minor-change-adding-a-defaulted-item

So allowed despite the breakage.

[joshtriplett]

I think it's still worth keeping this explanation, and just expanding it to make it clear that while we can make such a change, we may choose not to due to the possibility of breakage.

The explanation should also make clear that many traits we might want to add are especially likely to cause breakage, because they may conflict with similar traits from the crate ecosystem that exist to fill the same need that the standard library trait would.

[jyn514]

Another drawback: this would be ambiguous with std::io::Error.

[Lokathor]

yeah calling both types Error was a big flub :/

[Aloso]

@Lokathor it's a rather common design choice to reuse names in different modules. This means that you have to write io::Error instead of something like IoError, which I actually prefer. Another example is fmt::Write and io::Write. However, I don't like that the write! macro requires importing one of them, but that's another topic.

[Lokathor]

A common misconception. Only std::io and std::fmt do that weird thing where you're supposed to use the module name as a prefix. Every other error in the standard library just has a sensible name that doesn't need a prefix.

Either way, it's a bad plan at this point to put something called Error directly in the prelude.

It's possible that the confusion wouldn't be too high because one is a trait and the others are structs, but better to avoid the possibility.

[thomcc]

Why not add std::sync to the prelude like it was proposed for std::mem?

I think sync is probably as reasonable as the other proposed modules, but I honestly think you could make that argument for almost every module in the stdlib. I htink for that it's a compelling example of where having some sort of policy here would help.

(EDIT: When I replied to this it was in the thread about the std::sync changes... That said, that thread still seems out of order, so maybe github will fix itself?)

[SimonSapin]

I don’t think it’s useful to blame users for following the standard library’s pattern of io::Error and fmt::Error.

[Lokathor]

True, but the standard library has since moved away from that pattern (AllocError, Utf8Error, etc).

[CryZe]

AllocError is not stable and there's an open point about renaming it. Utf8Error is not in an utf8 module, so it's not changing any conventions. I believe I've heard some Error was accidentally stabilized not following that convention though. But I'm pretty sure there's no real precedent for changing that convention.

[SOF3]

Personally I find it annoying that I have to write std::sync::atomic::{AtomicU8, AtomicU16} instead of use std::sync::atomic and then atomic::U8 and atomic::U16 (it's even worse to write atomic::AtomicU8 and atomic::AtomicU16). I think it is quite idiomatic in the current state of Rust that we always prefix external items with their module (or crate if it's top-level), similar to the package.Item convention in golang (in which you can't even avoid that).

[thomcc]

Yeah, I kind of wish atomics were just in std::sync::AtomicU8 instead of std::sync::atomic::AtomicU8. I don't think atomic::U8 would be an improvement though, honestly I feel that would be worse. (This isn't on-topic though. These discussions get long enough as is)

I think it is quite idiomatic in the current state of Rust that we always prefix external items with their module

I don't think this bears out in practice. Most of the code I see and interact with isn't written this way. Additionally, autoimport functionality in IDEs doesn't work this way.

[shepmaster]

leftover?

[Manishearth]

Speaking from experience from the 2018 edition, this is extremely underspecified.

https://github.com/nikomatsakis/rfcs/blob/edition-2021-or-bust/text/0000-edition-2021.md#tooling-workflow

I'm going to leave a comment there as well, but each edition RFC that has breakages should have a section that:

• Lists out all the potential breakages, big and small
• Has an in-depth design for a migration lint that will migrate code such that it compiles on both editions. Make sure to get sign off from the compiler team that such a lint is feasible
• Optionally has an in-depth design for an idiom lint that will clean up the migrated code (potentially making it compile on the new edition only)

The 2018 edition RFCs tended to handwave about migration, and when the time came to implement it I realized that there wasn't actually a workable plan and as an implementor I had to come up with it at the last moment, trying to quickly build consensus. We should strongly avoid this situation recurring.

[leo60228]

File doesn't seem very useful without Read or Write. It also has the same problem with async as Mutex and RwLock.

[roblabla]

Though on that note, having Read/Write in the prelude would make a lot of APIs a lot easier to use. Socket and File IO are frequently annoying to use due to having to import both the traits and the concrete type (File/TcpStream).

[myrrlyn]

I would second adding Read/Write to the std::prelude, yes

[workingjubilee]

Maybe Read/Write but actually not the types? Unsure considering the AsyncRead concern, but

[Aloso]

If std::io::Write is added to the prelude, using std::fmt::Write becomes more difficult because of name conflicts. Not sure how many people use that trait, tho.

[coolreader18]

They could be added as use std::io::{Read as _, Write as _};

[saethlin]

If the prelude includes std::io::Write, how do I write! using std::fmt::Write to a String?

[ChrisJefferson]

Sorry to pop in with random questions, but would it be possible to provide evidence for these by looking at how often these types (and other std types) are used, and imported, over a large set of code (like cargo packages)?

[Manishearth]

I feel like modules are a pretty large change here, so far we've only ever had types, traits, and functions in the prelude, if modules are included then people may think they are crates. This drawback should be called out and addressed.

[myrrlyn]

Given that e2018 makes all cratenames implicitly present in every scope, I personally am very skeptical of having any other source of implicit inclusion of lowercase symbols.

Currently, the preludes only import one function: core::mem::drop. All other lowercase symbols imported by the preludes are macros, which can only be used with macname! or #[macname], and are not able to visually collide with crate, module, or function names in source. I tentatively would not disagree with placing additional functions in the prelude, but I firmly believe that placing module names in the prelude is a mistake for exactly the reason you outline.

[mbartlett21]

Should the RFC specify what happens if the module name collides with a crate? There are already mem and fmt crates (that have no reverse dependencies).

[Manishearth]

No, the prelude is just a glob import, glob imports have well-defined behavior around this.

[SOF3]

@mbartlett21 it's also possible that other packages explicitly specify the cargo lib.name to be mem or fmt. Not sure if there are any of those, but it's quite likely that this happens in private (as in not released on crates.io) monorepo projects. For example, when I write a binary that uses some ad-hoc procedural macros, I would create a my-codegen package in the same workspace with lib.name as a very short word, e.g. mac or gen. I am unsure whether other people would name their crates mem or fmt in a similar manner.

[bluss]

While adding mem/fmt to the prelude could be pragmatic, I think the drawback of introducing a "separate root" (apparently, effectively) like mem makes the whole thing harder to understand for users. I'd argue for lifting these functions to std::swap etc (as mentioned elsewhere) or other solutions, like no change.

[scottmcm]

A bunch of stuff for these is also applicable to other modules, like io::Result or ops::Add or whatever. Or one could argue for cmp::min to parallel with f64::min.

So I'm not convinced by the justification in here that specifically these two is the right thing to do.

[diondokter]

Maybe also mention whether or not this does/should affect the core prelude. IMO, the core prelude should contain the same things as the std prelude minus the std-specific stuff.

[sunjay]

Could you justify why this means that this module should be added to the prelude? I don't think that an item being imported a lot actually justifies adding it to the prelude. Especially given that we have no way of actually knowing whether "imported a lot" is true. (You can prove it for your code, and maybe for some of the code in public online, but there is so much more Rust code than that.) It is also worth discussing what we think the prelude should even be for. It's hard to discuss which particular items should go in before we've all agreed on that.

In general, I don't think saving people from writing something like use std::mem; is worth the confusion of people trying to figure out whether that name is an external crate or in the prelude.

I am reasonably in favor of adding some common traits like TryFrom/TryInto but types and modules should generally be imported explicitly so people can figure out where the name comes from when they read the code. The reason this is less of a problem for traits is because people generally call the trait methods themselves. They don't commonly refer to the trait name.

[myrrlyn]

You can prove it for your code, and maybe for some of the code in public online, but there is so much more Rust code than that.

We can, and in my opinion should (because I think it would be cool as well as educational), grep crater for the proportion of Rust modules that use symbols from the distribution libraries, whether by use or by fully qualified path at the use site.

[sunjay]

Yes, I guess my point is that not only is that data not complete (because it ignores a huge swath of non-open source code), it also embeds the assumption that usage is sufficient justification. We'd need to define what the prelude is actually for before we get there.

[myrrlyn]

because it ignores a huge swath of non-open source code

In the absence of raised evidence to the contrary, I think that assuming closed-source code has the same statistical shape as open-source code, especially given the volume to which crater has access, is a fairly safe thing to do.

A clearer philosophical decision about what the prelude is and should be is certainly worth doing regardless of whether its contents change.

[thomcc]

In the absence of raised evidence to the contrary, I think that assuming closed-source code has the same statistical shape as open-source code, especially given the volume to which crater has access, is a fairly safe thing to do.

I think this is probably not that true, since AFAIK crater has overwhelmingly more library crates than binaries. My experience is that this distribution isn't the same for private code.

That said, I don't know that we can do much here. At the end of the day, we use the data we have access to.

[Manishearth]

I feel like a thing that's missing here is a general policy for what we consider prelude-worthy. We don't have one right now, and my gut feeling is that this RFC goes many steps too far in expanding the prelude. Having an actual policy we can verify proposals against seems to be better than having a grab bag of things that people think should be in the prelude.

[workingjubilee]

Yes, it would help future edition changes make choices about the prelude, even if it's just a ponce... a policy only used once, I mean. We should be later able to revisit and see if the rationales were Bad, Actually. ( I am speaking from an assumption that, by sheer weight of inertia, people will think additions are good and subtractions are bad, so there will be a felt need of active justifications to undo a decision but not to make further expansive decisions. )

[scottmcm]

👍 to this. Without an agreed philosophy I suspect this will get bogged down substantially by all the details, since every single possible edition addition (doh!) could be its own long conversation.

Operationally, to get it in for the edition, I think it would make sense to focus this to a small number of minimally-controversial changes that very clearly fit the philosophy and need to be edition tied. Then future RFCs can be about specific additions to focus the conversation about them.

[Manishearth]

I think it would make sense to focus this to a small number of minimally-controversial changes that very clearly fit the philosophy and need to be edition tied

Strong agree.

I'd imagine this goes along the lines of:

• Figure out a policy (disallow discussing individual items until we have one, perhaps revert to pre-rfc until we have one)
• Figure out items which the policy clearly applies to
• People can disagree with the policy or disagree that the policy applies to an included element.

[kornelski]

I'm not sure if cmp::min/max should be elevated like that. I thought they were closer to being deprecated.

• These functions don't work with floats, and min(f, 1.0) is a gotcha that may surprise novice users. This makes non-prelude partial_min/partial_max look even more cumbersome in comparison.

• There's already Ord::min/max and same methods for floats, so these operations are already available without a need to import anything, and with a more consistent syntax. Inherent min/max are a bit odd when used for clamping, but there's Ord::clamp now too. And clamp doesn't have a cmp equivalent.

[mbartlett21]

Caveat: The fmt method will collide where the user originally only imported one of the traits.

E.g:

use core::fmt;
use core::fmt::Display;

struct MyStruct(u32);

impl Display for MyStruct {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        self.0.fmt(f) // Just delegate to the inner impl
    }
}

[djc]

I'm not sure I understand what you're getting at. That code seems to compile just fine?

[mbartlett21]

@djc

If you have a prelude that then implicitly imports Debug, this will then fail:

// The prelude with Debug:
mod prelude {
    pub(super) use core::fmt::Debug;
}

// Prelude import
use prelude::*;

use core::fmt;
use core::fmt::Display;

struct MyStruct(u32);

impl Display for MyStruct {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        self.0.fmt(f) // Just delegate to the inner impl
    }
}

(playground)

[kornelski]

If we're adding std modules (but I'm not sure if we should), I'd add io and fs. I use io::Result and fs::read quite often. When fmt::Result is available, it makes sense to have io::Result too.

[casey]

I like a lot of what this RFC proposes, but I think it might be easier to split it into multiple RFCs.

I think it would be easier to think about and discuss trait additions separately from other additions, since trait additions would be 2021-specific, and have separate concerns.

Additionally, I think that the non-trait additions (Path, HashMap, etcetera) are probably best addressed in dedicated RFCs for each addition or closely related group of additions (Path and PathBuf). Even though all things being equal, more RFCs means more work, I think addition-specific RFCs would be more streamlined, and the discussion would likely be more focused and productive.

As an example, I think I personally would like Path and PathBuf to be added to the prelude, since I use them so much, but would probably find it confusing if std::sync::Mutex were added, because sometimes I want to make sure I'm using an async version. If these were broken into separate RFCs, then they could be considered for inclusion separately, instead of having to reach consensus on a broad set of additions in order to merge the RFC.

Also, each prelude-addition RFC could be very streamlined, so doing an RFC for each might not be so much extra work.

[Lokathor]

I agree that while the traits are all an easy "yes", the rest is a lot of "oh, uh, maybe... not?"

[scottmcm]

Are very commonly used, not least of which in script-like CLI tools.

I wonder, for things like this, whether it would make sense for more of the WGs to define their own "sub-preludes" in crates, like for async things there's https://docs.rs/futures/0.3.13/futures/prelude/index.html

A CLI one of those could be great, for example, and be more aggressive about including lots of little things. Like I don't think create_dir makes sense in the global prelude, but would plausibly be fine in a scripting-focused one.

And that would avoid the obvious objections from other domains that just never use files, or don't want Path imported because they want Path to be an HTTP path rather than one that works differently on different operating systems.

[comex]

While most users will never see it, it's still odd to have the modules be named v1 and edition2021. Is it viable to rename v1 to edition2015 and have edition2018 and v1 (for compatibility) as aliases?

[djc]

I've updated this to explain the plan a bit more, see also the initial implementation PR. Does that make things sufficiently clear?

[kennytm]

???

use std::iter::FromIterator;
use std::collections::HashMap;

fn main() {
    dbg!(HashMap::from_iter(vec![(1_u64, 2_u64), (3, 4)].into_iter()));
}

Compiling this gives an error:

error[E0282]: type annotations needed
 --> src/main.rs:6:10
  |
6 |     dbg!(HashMap::from_iter(vec![(1_u64, 2_u64), (3, 4)].into_iter()));
  |          ^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `S` declared on the struct `HashMap`

error: aborting due to previous error

So no you still need to write HashMap::<_, _>::from_iter(iter) to set S = RandomState.

[Kestrer]

Sorry to pop in with random questions, but would it be possible to provide evidence for these by looking at how often these types (and other std types) are used, and imported, over a large set of code (like cargo packages)?

@ChrisJefferson I collected some evidence about this a while ago, if it helps.

[matu3ba]

1. How does this affect compile times for programs that do not use these features? I am slightly worried, that the user pays for things that the user does not use.

2. I do not understand the usability issues of missing includes, when we have rust-analyzer/the language server suggesting them (for std at least).

3. I do not understand nudging people into not using bad methods, when we could a.deprecate the method b.moving the method into a better understandable name c.group methods together to do a.+b.

Could you shortly elaborate and clarify what I am missing?

[thomcc]

1. How does this affect compile times for programs that do not use these features? I am slightly worried, that the user pays for things that the user does not use.

Trivially if at all. This is not a concern here.

2. I do not understand the usability issues of missing includes, when we have rust-analyzer/the language server suggesting them (for std at least).

The language shouldn't rely on people using r-a. Also, this won't help in all cases. For example, the cases like the Try{From,Into} traits, which are the most inarguable thing worth including here IMO.

[ChrisDenton]

The motivation/drawbacks talks about adding traits but doesn't justify adding types. I'm not wholly against adding types to the prelude but I do feel they need more discussion. What are the downsides? If there are none, why aren't all std types in the prelude so long as they don't clash with each other? How much of an inconvenience is importing types? Could this inconvenience be reduced in other ways? Reading through the explanations given for each item, I think this sums up the various motivations:

• other languages have it in their built-ins
• is commonly used
• it's useful in script-like tools
• unlikely to clash with other types
• types in the prelude should not cause issues since explicit imports will always override prelude imports

I think it would be great if these had more discussion. I wonder if putting more types in the prelude deserves a separate RFC? Especially as the motivation says adding types does not need an edition boundary.

[mbartlett21]

Something that I didn't have a full understanding of until the libs team meeting last night: the following traits are part of the v1 prelude but are doc(hidden) because of a rustdoc bug: Clone, Copy, Debug, Default, Eq, Hash, Ord, PartialEq, PartialOrd. [...]

The Debug and Hash imports are just the derive macros, not the actual traits.

[djc]

Given that FromIterator does not work as advertised in the RFC, what is the motivation for adding it at all? At least, the RFC should be updated to fix the bad example.

It doesn't work for that particular example with the three-argument HashMap. It would still apply to Vec or alternative HashMap implementations that don't have the hasher type argument (or really, any implementer of FromIterator that has some type arguments). I'll update the example.

[scottmcm]

For things like replace, this conversation just happened on the community discord's beginner channel:

I don't know if people would feel less bad about it if it was in the prelude, but maybe...

[djc]

Started a std/core PR for the edition traits in rust-lang/rust#82781. This failed pretty quickly because the minifier crate which rustdoc appears to rely on defines a custom MyTryFrom -- submitted a fix for that in GuillaumeGomez/minifier-rs#65.

[SimonSapin]

@djc We’ve tried this before when stabilizing these traits in rust-lang/rust#49305, conclusively established that breakage is too widespread in this case, and reverted in rust-lang/rust#49518. This is the entire reason we’re discussing edition-specific preludes.

I don’t understand the point of rust-lang/rust#82781, and especially of GuillaumeGomez/minifier-rs#65. Forcing the entire ecosystem through this transition would be a lot more painful than it’s worth.

[Kixunil]

@scottmcm it was already discussed that there doesn't seem to be strong connection between knowing some part of Rust and being in prelude. If you want to find out the truth, ask the person these questions

• Why it looks dangerous?
• What should be done to make it look not dangerous?

Frankly, I suspect the person doesn't know that there's unsafe Rust clearly marked by the keyword so if there's no unsafe it's not dangerous (but yes, [i] may panic). I'd love to see my hypothesis invalidated, though. :)

[QuineDot]

Frankly, I suspect the person doesn't know that there's unsafe Rust clearly marked by the keyword so if there's no unsafe it's not dangerous (but yes, [i] may panic). I'd love to see my hypothesis invalidated, though. :)

I related the same experience on the IRLO thread.. To save people the click, my uncertainty came from some combination of not seeing the pattern before and it not being imported while living next to things like transmute and forget. I did also know about unsafe, although (being a beginner) I probably didn't fully grok the full boundaries of it. I didn't detail this part in that IRLO post, but in my (pre-NLL) grapples with the borrow checker, I had already used forget (a safe function) in ways that I eventually figured out were poor choices. In other words, I had already learned that the fact a function is not unsafe does not mean it's a good idea (or not dangerous). (I had not seen any material that covered the utility of drop vs forget, either; reading the documentation of forget now, it seems okay, though maybe it could emphasize drop even more. I don't remember specifically what it was like then though.)

I haven't reread the book in awhile, but at least at the time, these weren't scenarios covered by the introductory materials. When it comes up on URLO today, I'm likely to suggest this article by Niko instead, and/or just suggest swap, replace, or take. This is the sort of shortfall in discoverability I was referencing in my previous comment. I don't know if adding to the prelude is the answer either, but the fact that it still throws off beginners indicates that it's still a problem to me.

[Kixunil]

@QuineDot thank you for thoughtful comment!

living next to things like transmute and forget

This could be used as justification for adding it to prelude. I'm still not sure, but at least it's something interesting. Of course this can be also solved by introducing new module(s) or just writing better doc.

Actually the article you point to seems to describe what is the most important use of swap, replace, and take, so maybe having a short version of it in docs would make sense.

[djc]

@djc We’ve tried this before when stabilizing these traits in rust-lang/rust#49305, conclusively established that breakage is too widespread in this case, and reverted in rust-lang/rust#49518. This is the entire reason we’re discussing edition-specific preludes.

Consensus in the libs team meeting was that the breakage might have been reduced in the 3 years since the previous attempt, since these traits have now been stable in std for quite a long time, and that it might be worth assessing the amount of damage. In addition, the PR also adds the FromIterator trait which, as far as I know, hasn't been tried before.

I don’t understand the point of rust-lang/rust#82781, and especially of GuillaumeGomez/minifier-rs#65. Forcing the entire ecosystem through this transition would be a lot more painful than it’s worth.

Are you saying we should not add these traits to the prelude at all, or that we shouldn't bother assessing the breakage and just focus on making sure lints/cargo fix provide a good experience?

[lunabunn]

I don’t understand the point of rust-lang/rust#82781, and especially of GuillaumeGomez/minifier-rs#65. Forcing the entire ecosystem through this transition would be a lot more painful than it’s worth.

The first one says this:

This is a test PR which I'd like to get a crater run for, as input to the 2021 prelude RFC. Please don't merge!

The second one seems like:
a) a test to see what kinds of compat issues might arise if those changes were merged to the 2021 edition prelude
b) prep to migrate

I don't think anyone's saying we should force the entire ecosystem through anything.

[kornelski]

Some alternatives to adding new types or functions to the prelude:

• 1 pub use types and functions at the top level of std, like std::HashMap, std::swap. This way these items would still be practical to use without importing them.

• 2 Add language features that allow users to crate their own preludes, so that each project can choose how much or how little they want globally available.

[jyn514]

Something that I didn't have a full understanding of until the libs team meeting last night: the following traits are part of the v1 prelude but are doc(hidden) because of a rustdoc bug: Clone, Copy, Debug, Default, Eq, Hash, Ord, PartialEq, PartialOrd. It was suggested that these might have "accidentally" been added to the prelude when the derive implementations were reimplemented as regular procedural macros. Also, AsRef and AsMut are already in the prelude, which I missed earlier.

Repeating my comments from discord: this may have been fixed by rust-lang/rust#77862, but I don't know without testing. If someone wants to make a PR removing doc(hidden) and seeing if that works, that would be very helpful.

[m-ou-se]

@jyn514 checking now if it works.

[kornelski]

How about removing RustcEncodable?

[djc]

Sorry for the lack of updates here. As suggested by @m-ou-se on Zulip, I've split out the trait changes from this RFC and moved them into a smaller separate PR, RFC 3114. Will likely focus on finishing that up first before I circle back to the type parts of this RFC.

[jhpratt]

This should probably be closed unless there's desire to use this PR for 2024?

[djc]

As I understand it, the non-trait changes do not need to wait for an edition change, so we could just move forward with the remaining changes (also some further discussion and RFC updates are warranted). I'll see if I can get to it soon -- if someone else wants to take over, that'd be fine too.

[tgross35]

After talking with Josh on zulip would you maybe want to update the title & doc to indicate we want to reuse this issue for the 2023 edition? If you are still interested in maintaining this PR.

[djc]

I don't think it makes sense to reuse this for 2024 edition, given that most of the trait contents (which are the only parts that really need an edition as I understand it) moved into RFC 3114. I'll just close this for now, since I am not currently motivated to work on this.