RFC: absolutely minimal way to close #18823

[StefanKarpinski]

I think that this would be far less disruptive than the proposal in #18823, not to mention that it's getting too late to make such a huge change to the iteration protocol at this point. This allows iterators that don't know they're done until they've tried to return nothing from the next method instead of a value, state tuple and return early that way. There's no impact on iterators that don't need to do that and there's no impact on code that uses the iteration protocol to process generic iterators as long as they don't need to work with "unforseeable iterators". So any code that works for arrays, strings, etc. needs no changing. Generic code that needs to work with these kinds of iterators can be fixed gradually, as we encounter it failing in the wild. What does need to be done in addition to this minimal changes is to make all the iterators in Base that should use this functionality actually use it.

This approach does have a few issues, but which I believe it shares with the proposal in #18823:

• The generic isempty definition seems wrong now since it checks done(itr, state) which for this kind of iterator will return false, yet the following call to next can return nothing leaving one with no values, despite isempty being false. Perhaps this is ok if we document that for such iterators the isempty predicate can be unreliable: the iterator may not know that it's empty, yet not return any valules.

• There are probably other generic uses of done like the definition of isempty that would have similar problems.

• @JeffBezanson has expressed some concerns about the expansion of code involved in the new lowering. It doesn't emit that much extra code, but for loops are very common so it could still be a problem.

[StefanKarpinski]

• Win32 run succeeded but then timed out on some network stuff
• Win64 timed out on early network stuff
• Travis Linux runs both succeeded

[stevengj]

I assume this check gets optimized out in the common case of a next function that always returns a tuple?

Hopefully it gets optimized out even for iteration over a type-unstable container, e.g. where next returns Tuple{ ...inference fails... , MyState}?

[JeffBezanson]

From triage: we'll try to implement the full new iteration protocol and see how far we get. The protocol can include both iterate and optionally done. If an iterator is able to know that it's done without advancing, it can implement done, and otherwise not implement it.

[StefanKarpinski]

Since I goaded Keno and Jameson into opening #25261, my work is done here.