[Transform] Latest transforms may disregard some document updates.

[przemekwitek]

Elasticsearch Version

8.2.3

Installed Plugins

No response

Java Version

bundled

OS Version

n/a

Problem Description

For each order the user wants to see the latest status of this order.
Therefore they use continuous latest transform with:

• unique_key: orderId
• sort: updatedAt
• sync.time.field: event.ingested (they've also tried using @timestamp instead)

Unfortunately sometimes in their dashboard they see the order's old status instead of the order's new status.
The suspicion is that this happens if:
document_v1.updatedAt < document_v2.updatedAt but document_v1.event.ingested > document_v2.event.ingested2

The problem is that the latest transform requires that the sort field values increase monotonically over time, just like the sync.time.field field values.
This GH issue is about relaxing this restriction so that the document ingested later (but with earlier value of the sort field) does not overwrite the document with later value of the sort field.

Steps to Reproduce

Described in the external support issue. Will not copy the steps here for now.

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)