RFC: make countlines count final non-empty line even without EOL

[stevengj]

This is a breaking change that makes countlines agree with the length of eachline and readlines when the last non-empty line does not end with EOL.

See the discussion on discourse.

[StefanKarpinski]

We might want to have some way of preserving the old behavior. One way would be to consider the default eol to be a regex like r"\r?\n|$" – even if we implement it more efficiently. That would make the old behavior equivalent to having eol = '\n' and so on. We could also have a partial = true keyword option to control whether a trailing partial line is counted or not.

[stevengj]

When would you ever want countlines(io) to not equal length(readlines(io))?

[StefanKarpinski]

Note that if we didn't allow specification of what constitutes end-of-line this would be straightforward since it seems pretty clear that countlines should agree with counting lines yielded by eachline. The eol argument complicates matter, however, since we have to decide whether eol = '.' counts a trailing sentence with no period, for example. There are some nice benefits of allowing a regex for eol, such as potentially supporting the old Mac newlines with a regex like r"\r?\n?|$" – that would count \r and \n by themselves as an end of line but count consecutive \r\n as a single line terminator. Now, I'm not sure we should actually make that a default, but at least supporting it is nice.

[stevengj]

In the rare cases (if any) where someone needs the old behavior, can't they easily get this via:

function countnontrailing(io, eol)
    n = countlines(io)
    return n == 0 ? n : n - (read(skip(seekend(io),-1), UInt8) != eol)
end

That would probably be lot more efficient than using a regex.

[Keno]

I feel like we could at some point have a Char iterator over files and write the current behavior as:

count(equalto('\n'), eachbyte(file))

so 👍 on this change.

[StefanKarpinski]

Triage is in favor of accepting this as-is (I agree).

[stevengj]

Probably eachread(UInt8, io) would be a more useful file-iterator interface, so that you could read bytes, chars, whatever.