support UInt & BigInt in TOML

[Roger-luo]

This partially implements @StefanKarpinski's option 2 here: JuliaLang/TOML.jl#46 (comment), with the following convention

• all numbers below UInt64 and Int64 are parsed as UInt64 or Int64 for compatibility
• numbers above UInt64 and Int64 will be parsed accordingly to 128 or BigInt

resolves JuliaLang/TOML.jl#46

cc: @KristofferC

[Roger-luo]

Not sure if the test failure is due to this PR? The TOML tests pass locally for me

[StefanKarpinski]

This is getting there, but I think if we're going to use native Julia types to represent integer literals, we should go all the way and use the same native types to represent integer literals as we would in Julia. In other words:

• Decimal literals from -2^63:2^63-1 are Int64
• Decimal literals from -2^127:2^127-1 are Int128
• Larger decimal literals are BigInt

For binary, octal and hex literals, the type is determined by the number of digits the literal has (not the value of the literal, but the number of digits, d, that it is written with):

• Binary literals are UInt$n where n = bits(2, d)
• Octal literals are UInt$n where n = bits(8, d)
• Hex literals are UInt$n where n = bits(16, d)

Where bits(b, d) = 8*nextpow(2, d*log2(b)/8). Except that UInt$n is replaced by BigInt once n is greater than 128 since we don't have unsigned types larger than that.

[vtjnash]

I am not sure the type you get back should be quite that closely dependent on the file content. I would think Int/UInt/BigInt should suffice and be cleaner for the API. It is not quite like parsing source code, where the semantic intent is being conveyed intentionally also by the code length.

[KristofferC]

I am not sure the type you get back should be quite that closely dependent on the file content. I would think Int/UInt/BigInt should suffice and be cleaner for the API. It is not quite like parsing source code, where the semantic intent is being conveyed intentionally also by the code length.

Yes, having a whole big range of types just feels like it would cause more compilation etc for very little value. Agree with Int/UInt/BigInt and that's it.

[vtjnash]

Edit: I think the spec says Int64/UInt64, but yes

[KristofferC]

Spec only says at least Int64.

[Roger-luo]

Yes, having a whole big range of types just feels like it would cause more compilation etc for very little value. Agree with Int/UInt/BigInt and that's it.

should I remove the support of UInt128 and Int128 then?

---

I think the other reason we cannot support Uint8 etc. just that this will be quite breaking (they used to be returning Int), but UInt etc errors occasionally after serialization from Julia, so this is slightly less breaking I think.

[StefanKarpinski]

Well, it would only be breaking for unsigned integer literals, which I suspect are not widely used in our world. The point of using different types is to use the type as a way to preserve more information about how to print values (i.e. how many digits they need). If we're going to to Int64/UInt64/BigInt and print UInt64 as hex, then I'm not sure we want to use Julia's printing for those values. If I round trip a TOML file with 0xff in it and get back 0x00000000000000ff in my file, that's pretty annoying. So in that case, you've got to skip the leading zero bytes (pairs of digits rather than digits seems best).

[StefanKarpinski]

I also think that keeping the Int128 and UInt128 options here seems reasonable since those types are so much more efficient than BigInts. But I don't feel strongly about that either.

[Roger-luo]

OK, so I kept the original parsing, where

• all integers below 64-bit are parsed into Int64 or if written in bin,oct and hex will be parsed into UInt64
• integers between 64-bit - 128-bit are parsed as Int128 or UInt128
• integers larger parsed as BigInt

for printing, signed integers are printed via Base.show (same as previous), unsigned integers
are printed as even hex without leading zeros.

---

I copy-pasted log2i from Yao to implement that even number of hex printing, it returns an integer strictly when taking an integer, I find it quite convenient when working with bits, but I'm not sure if people want me to move it into Base and let it sit beside log functions (in a separate PR)? It's just some convenient one-liner.

[Roger-luo]

@StefanKarpinski bump, do you have any other comments? Can we merge this?

[StefanKarpinski]

Looks good although I'm not seeing where this tests for printing of unsigned values.

[StefanKarpinski]

LGTM

[KristofferC]

Thanks for this @Roger-luo