Deleted records returned when using equality deletes and filtering on columns not part of the Iceberg Identifier Field IDs

[nrutherford-w]

Description

Trino queries return deleted records. This occurs when using equality deletes with Iceberg and filtering by columns that are not part of the Iceberg Identifier Field IDs.

Steps to Reproduce

1. Create Table

CREATE TABLE orders (
   order_pk integer NOT NULL,
   customer_id integer,
   order_void boolean
)
WITH (
   format = 'ORC',
   format_version = 2,
   location = 'dir/orders-1c20829da05e477893e047e4ba8034e0',
   partitioning = ARRAY['order_pk']
)

2. Inset into orders table

order_pk	customer_id	order_void
19	400	0
20	400	0
21	400	0
22	400	0

3. Delete from orders table

order_pk
21
22

4. Select from orders table
   SELECT * FROM orders

Expected response is received

order_pk	customer_id	order_void
19	400	0
20	400	0

5. Select from orders table filtering by a column that is not part of the table's partitioning
   SELECT * FROM orders WHERE NOT order_void

Response includes deleted records

order_pk	customer_id	order_void
19	400	0
20	400	0
21	400	0
22	400	0

Expected Result:

The query SELECT * FROM orders WHERE NOT order_void should not return records with 'order_pk' 21 and 22 as they were deleted.

Actual Result:

The query SELECT * FROM orders WHERE NOT order_void returns deleted records with 'order_pk' 21 and 22.

Additional Information

Trino Version 449
Inserts and equality deletes are written through the Iceberg API.
Running insert and delete statements in Trino will not recreate the issue.

Attachments

Insert orc files for order_pk = 22
989149293-1-94f64dda-537c-4ada-ab9b-4b1c1bdc64ed-00007.orc
989149293-1-94f64dda-537c-4ada-ab9b-4b1c1bdc64ed-00008.orc
Delete orc file for order_pk = 22
989569443-1-bfa480a1-c1ee-4982-9d55-176a46cac1a2-00004.orc

orc-files.zip

[ebyhr]

@nrutherford-w I can't reproduce the issue on master. What's your Trino version? Also, please share the actual steps to reproduce without converting statements to markdown tables.

[nrutherford-w]

@ebyhr Trino version is 449. Insert and Deletes are processed through the Iceberg API. Attached to the initial comment are the .orc files from the insert and delete for order_pk 22

[ebyhr]

@nrutherford-w Thanks for sharing the file. Please share the code snippet so that we can reproduce easily.