DateTime + Nanosecond loses information

[jariji]

This issue came up on Slack.

julia> dt = now()
2023-08-03T10:01:51.027

julia> dt == dt + Nanosecond(10^6 - 1)
true


julia> dump(now())
DateTime
  instant: Dates.UTInstant{Millisecond}
    periods: Millisecond
      value: Int64 63826748639606

DateTimes are integral so I would expect this to promote or error, not fail silently. Floating point can lose data, but I don't think integral types do this in Julia.

[Moelf]

That's because DateTime only tracks things in millisecond and

julia> Dates.toms(Nanosecond(10^6))
1

julia> Dates.toms(Nanosecond(10^6 - 1))
0

you can see we always add things in ms:

julia/stdlib/Dates/src/arithmetic.jl

Lines 82 to 83 in 6dd763b

	(+)(x::DateTime, y::Period) = return DateTime(UTM(value(x) + toms(y)))
	(-)(x::DateTime, y::Period) = return DateTime(UTM(value(x) - toms(y)))

[jariji]

toms is a private implementation detail. Integral types don't lose information like that. They promote or error.

julia> 1 + eps(0.0)
1.0

julia> 1 + eps(0.0) === 1
false

[Moelf]

That's not a good analogy, DateTime can't use Nanosecond internally without using Big number, so it's actually pretty fundamental we go to ms

[jariji]

Converting to milliseconds may be essential if being quantized in milliseconds is part of the API of DateTime. But the current behavior of Dates.toms is not part of that API - that function is undocumented and its use is not part of the API of the DateTime constructor or Base.+.

If I'm converting into T a value that's not exactly representable in T, Julia typically promotes, as in the float case -- or if it can't, it gives an InexactError. Julia doesn't generally throw away otherwise accurate values.

DateTime can't use Nanosecond internally without using Big number

If DateTime doesn't support what the user is asking for, it should say so, and the user can install NanoDates.jl.

[barucden]

The documentation could be more explicit about DateTime's resolution. Right now it only says "DateTime wraps a UTInstant{Millisecond} [...]".

Also, when constructing from a Date and a Time, DateTime does throw an InexactError:

julia> DateTime( Date(now()), Time(Nanosecond(1)) )
ERROR: InexactError: DateTime(00:00:00.000000001)

This behavior is explicitly implemented as well as documented.

Maybe toms should throw an InexactError when its argument cannot be expressed as milliseconds (i.e., Nanosecond and Microsecond not divisible by 10^6 and 10^3, respectively)? Although, that would probably be breaking.

[PallHaraldsson]

That's because DateTime only tracks things in millisecond

Even if, should:

julia> x === x + Nanosecond(10^6 - 1)
true

[Moelf]

Yes? Unless you want to error on DateTime + Nanosecond.

If you're routinely adding less than 10^6 nanosecond, you're probably not using Julia DateTime built-in stuff anyway

[barucden]

A test was added in #30912 to check that toms truncates nanoseconds. A similar test was added in #32037 for microseconds. So changing the corresponding methods of toms to

toms(c::Nanosecond)  = divexact(value(c), 1000000)
toms(c::Microsecond) = divexact(value(c), 1000)

breaks even our test suite.

[jariji]

Again, toms is a private, internal function, not part of Julia's API, so its behavior under any particular version is not restricted. Nor does + need to call toms.

---

Here's an example of Julia correctly refusing to lose information:

julia> today()
2023-08-07

julia> today() + Hour(1)
ERROR: MethodError: no method matching +(::Date, ::Hour)

For now, deprecating without removing DateTime + Nanosecond would be a good solution, IMO.

[Seelengrab]

This has not been fixed by #50816, I don't think. We're now inventing information out of thin air instead of losing it, by sometimes rounding up.

EDIT: and we're also still losing it, by rounding down. This means there are now worse precision problems than before.

[timholy]

(I agree this hasn't been closed by #50816, that's the part the thumbs-up is for.)

[jariji]

Related: #35446