CRLF support for file formats other than CSV

[Komosa]

Currently parsing files with CRLF line ending is not supported. (I used simplest possible DKVP file cat'ed to mlr put)
I know, that now you are working on CSV-RFC4180 support (which explicitly says that CRLF is vaild (and required - :( ).

For now, I can suggest following:
0. at least log warning if CRLF is encountered.

1. silently drop CRLFs

Later of course proper support for line endings should be added :)

BTW. really great tool!

[johnkerl]

thanks! yes, rfc-csv currently #1. minimally, adding an option to do CRLF rather than LF is feasible. as an aside, depending on how the technical details work out i may replace my current pointer-walking file parsers with a lemon parser (currently i only use a lemon parser for DSLs, not file data) in which case we could get double-quoting in other formats besides CSV.

[ve3ied]

Or, stick tr out front for now.

tr -d '\015' | mlr ...

[johnkerl]

Marking as wishlist for now, just relative to other things more pressing -- namely, RFC-CSV and packaging. In the medium term, though, this will be a should-do rather than a wishlist item.

[johnkerl]

Dup'ing to #50

[johnkerl]

Change of plans ... multi-character-separator options for CSV is separately implemented on #50. Meanwhile doing that for other formats -- and doing it performantly (RFC-CSV I/O is already slow, and I don't want to make the other formats also slow) -- is a separate, still-open coding task.

[johnkerl]

So, I'm re-opening this issue.

[johnkerl]

Support now exists for DKVP, e.g. mlr --rs crlf cut -f a,x,y myfile.dkvp, or mlr --ifs ': ' --ofs =, etc.

Please let me know if you find a bug and I'll add it as a unit-test case (in addition to fixing the bug of course).

I'll leave this issue open until other formats also have multi-character support.

[johnkerl]

Multi-char-separator CSV-lite done

[johnkerl]

Resolved by https://github.com/johnkerl/miller/releases/tag/v2.2.0