Seq.shuffle() (and others) should delay buffering until a terminal op requests the stream values

[lukaseder]

See comment there:
aol/cyclops#99 (comment)

Others:

• innerJoin()
• leftOuterJoin()
• removeAll()
• retainAll()
• window()
• zip() eagerly allocates an Iterator
• SeqUtils.transform() eagerly allocates a Spliterator
• reverse()
• crossJoin()
• duplicate()
• grouped()
• partition

In short: Each operation has this flaw

[tlinkowski]

An idea on how to get rid of toList() calls in some of the methods mentioned above.

I think of a special SeqBuffer class (possibly implementing Streamable [#85]) that would decorate given Seq<T>. Internally it would use this Seq's specially wrapped Spliterator<T> spliterator. This Spliterator would add consumed elements to an ArrayList<T> buffer. And the seq() method of this SeqBuffer would return Seq.seq(buffer).append(spliterator).

In every place we now call toList we could create an instance of this class, and then call its seq() method.

But it's just a theoretical concept from the top of my head - I haven't tested this in any way and I don't know whether this would really work.

[tlinkowski]

No, it is a bad solution. The above wouldn't work if the seq() method were called more than once and then both Seqs were consumed interchangeably:

1. Seq.seq(buffer).append(spliterator) would throw a ConcurrentModificationException
2. if both of the Seqs entered the append(spliterator) section, they would not return all the elements

But I'm pretty sure it can be done to support multiple seq() calls and arbitrary (but single-threaded) consumption of all the resulting Seqs.

[lukaseder]

Hmm, I'm happy to look at a concrete code implementation :)