Support thousands separator in `fmtnum` #1499

johnkerl · 2024-02-18T18:00:21Z

Needs documentation updates
Needs unit-test cases

johnkerl · 2024-02-18T18:44:44Z

@lulunac27a @aborruso here are some (draft) examples:

$ export LANG=en
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1,000,000,000 | 346,790,144.3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1,414,213,562 | 758,679,964.7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1,732,050,807 | 204,603,305.7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2,000,000,000 | 381,399,393.8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2,236,067,977 | 573,288,919.8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2,449,489,742 | 527,126,160.0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2,645,751,311 | 611,784,060.5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2,828,427,124 | 598,554,009.1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3,000,000,000 | 31,441,876.4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3,162,277,660 | 502,626,005.5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

$ export LANG=fr
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1 000 000 000 | 346 790 144,3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1 414 213 562 | 758 679 964,7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1 732 050 807 | 204 603 305,7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2 000 000 000 | 381 399 393,8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2 236 067 977 | 573 288 919,8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2 449 489 742 | 527 126 160,0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2 645 751 311 | 611 784 060,5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2 828 427 124 | 598 554 009,1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3 000 000 000 | 31 441 876,4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3 162 277 660 | 502 626 005,5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

$ export LANG=it
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1.000.000.000 | 346.790.144,3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1.414.213.562 | 758.679.964,7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1.732.050.807 | 204.603.305,7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2.000.000.000 | 381.399.393,8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2.236.067.977 | 573.288.919,8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2.449.489.742 | 527.126.160,0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2.645.751.311 | 611.784.060,5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2.828.427.124 | 598.554.009,1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3.000.000.000 | 31.441.876,4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3.162.277.660 | 502.626.005,5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

Idea:

The %'d is too hard to escape with shell quotes -- use %_d for Miller instead
Likewise %_f
Language is "en" by default; consults the (very common) LANG environment variable if available
I haven't written on-line doc materials yet

What do you think?

aborruso · 2024-02-18T18:52:01Z

This is really a great new feature. I have no notes to add, only thank you very much!!!

lulunac27a · 2024-02-18T23:03:28Z

Numeric format with fixed number of decimal digits should be right aligned, not left aligned by default (referenced from this code above.

johnkerl · 2024-02-19T00:38:48Z

@lulunac27a I'm confused. The examples above do exhibit right-alignment.

johnkerl · 2024-02-19T00:40:08Z

@lulunac27a I think I see -- ?

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%18_d")'
+-----+-----+----+---------------------+---------------------+--------------------+
| a   | b   | i  | x                   | y                   | si                 |
+-----+-----+----+---------------------+---------------------+--------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |                 10 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |                141 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |              1,732 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |             20,000 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |            223,606 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |          2,449,489 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |         26,457,513 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |        282,842,712 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3,000,000,000 |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |     31,622,776,601 |
+-----+-----+----+---------------------+---------------------+--------------------+

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")'
+-----+-----+----+---------------------+---------------------+-------------+
| a   | b   | i  | x                   | y                   | si          |
+-----+-----+----+---------------------+---------------------+-------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  | 10          |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  | 141         |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 | 1732        |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 | 20000       |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  | 223606      |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 | 2449489     |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  | 26457513    |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   | 282842712   |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  | 3000000000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  | 31622776601 |
+-----+-----+----+---------------------+---------------------+-------------+

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%_d")'
+-----+-----+----+---------------------+---------------------+----------------+
| a   | b   | i  | x                   | y                   | si             |
+-----+-----+----+---------------------+---------------------+----------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  | 10             |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  | 141            |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 | 1,732          |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 | 20,000         |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  | 223,606        |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 | 2,449,489      |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  | 26,457,513     |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   | 282,842,712    |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  | 3,000,000,000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  | 31,622,776,601 |
+-----+-----+----+---------------------+---------------------+----------------+

johnkerl · 2024-02-19T00:42:55Z

I think the issue involves how pprint output does alignment of numeric values:

$ mlr --ojson --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
[
{
  "a": "hat",
  "b": "wye",
  "i": 9,
  "x": 0.03144187646093577,
  "y": 0.7495507603507059,
  "si": 3000000000
},
{
  "a": "pan",
  "b": "wye",
  "i": 10,
  "x": 0.5026260055412137,
  "y": 0.9526183602969864,
  "si": 31622776601
}
]

$ mlr --ocsv --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
a,b,i,x,y,si
hat,wye,9,0.03144187646093577,0.7495507603507059,3000000000
pan,wye,10,0.5026260055412137,0.9526183602969864,31622776601

$ mlr --opprint --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
a   b   i  x                   y                  si
hat wye 9  0.03144187646093577 0.7495507603507059 3000000000
pan wye 10 0.5026260055412137  0.9526183602969864 31622776601

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
+-----+-----+----+---------------------+--------------------+-------------+
| a   | b   | i  | x                   | y                  | si          |
+-----+-----+----+---------------------+--------------------+-------------+
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059 | 3000000000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864 | 31622776601 |
+-----+-----+----+---------------------+--------------------+-------------+

johnkerl marked this pull request as draft February 18, 2024 18:18

johnkerl force-pushed the kerl/fmtnum-commas branch from fec4d10 to 1b19d11 Compare February 18, 2024 18:32

johnkerl force-pushed the kerl/fmtnum-commas branch 2 times, most recently from 5831afa to a60348c Compare February 18, 2024 18:51

johnkerl added 2 commits February 18, 2024 13:54

Support thousands separator in fmtnum

30f26b7

doc bits

73ecb0a

johnkerl force-pushed the kerl/fmtnum-commas branch from a60348c to 73ecb0a Compare February 18, 2024 18:55

johnkerl marked this pull request as ready for review February 18, 2024 18:56

johnkerl changed the title ~~Support thousands separator in fmtnum [WIP]~~ Support thousands separator in fmtnum Feb 18, 2024

johnkerl merged commit 7bd460a into main Feb 18, 2024
6 checks passed

johnkerl deleted the kerl/fmtnum-commas branch February 18, 2024 19:01

johnkerl mentioned this pull request Feb 18, 2024

Add support for thousands separator in fmtnum #1490

Closed

johnkerl mentioned this pull request Feb 22, 2024

[feature request] Right-align numeric values in PPRINT and Markdown output formats #1503

Open

chenrui333 mentioned this pull request Mar 20, 2024

miller 6.12.0 Homebrew/homebrew-core#166715

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support thousands separator in `fmtnum` #1499

Support thousands separator in `fmtnum` #1499

johnkerl commented Feb 18, 2024 •

edited

Loading

johnkerl commented Feb 18, 2024 •

edited

Loading

aborruso commented Feb 18, 2024

lulunac27a commented Feb 18, 2024 •

edited

Loading

johnkerl commented Feb 19, 2024

johnkerl commented Feb 19, 2024

johnkerl commented Feb 19, 2024

Support thousands separator in fmtnum #1499

Support thousands separator in fmtnum #1499

Conversation

johnkerl commented Feb 18, 2024 • edited Loading

johnkerl commented Feb 18, 2024 • edited Loading

aborruso commented Feb 18, 2024

lulunac27a commented Feb 18, 2024 • edited Loading

johnkerl commented Feb 19, 2024

johnkerl commented Feb 19, 2024

johnkerl commented Feb 19, 2024

Support thousands separator in `fmtnum` #1499

Support thousands separator in `fmtnum` #1499

johnkerl commented Feb 18, 2024 •

edited

Loading

johnkerl commented Feb 18, 2024 •

edited

Loading

lulunac27a commented Feb 18, 2024 •

edited

Loading