[VL] Performs partial sort before global sort in SortShuffleWriter

[marin-ma]

No description provided.

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[marin-ma]

@GlutenPerfBot benchmark

[GlutenPerfBot]

ACK, will benchmark TPCH/DS on this pull request

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_6160_time.csv	log/native_master_07_03_2024_a8e8700ef8_time.csv	difference	percentage
q1	34.62	34.56	-0.069	99.80%
q2	23.95	24.03	0.082	100.34%
q3	41.52	39.79	-1.731	95.83%
q4	32.86	32.60	-0.262	99.20%
q5	68.78	70.06	1.279	101.86%
q6	7.80	6.39	-1.408	81.94%
q7	81.91	82.44	0.531	100.65%
q8	84.53	85.16	0.625	100.74%
q9	119.29	121.52	2.232	101.87%
q10	46.64	47.44	0.796	101.71%
q11	20.60	20.52	-0.082	99.60%
q12	25.56	26.96	1.407	105.51%
q13	39.62	38.00	-1.625	95.90%
q14	18.89	21.40	2.517	113.33%
q15	35.17	31.78	-3.392	90.36%
q16	14.43	13.13	-1.295	91.02%
q17	105.07	104.52	-0.547	99.48%
q18	148.28	147.93	-0.344	99.77%
q19	14.96	13.84	-1.122	92.50%
q20	29.48	29.65	0.176	100.60%
q21	261.00	265.38	4.373	101.68%
q22	12.29	12.52	0.234	101.90%
total	1267.23	1269.61	2.376	100.19%

[github-actions]

Run Gluten Clickhouse CI

[GlutenPerfBot]

ACK, will benchmark TPCH/DS on this pull request

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_6160_time.csv	log/native_master_07_03_2024_a8e8700ef8_time.csv	difference	percentage
q1	37.08	34.56	-2.526	93.19%
q2	22.61	24.03	1.421	106.28%
q3	40.93	39.79	-1.139	97.22%
q4	33.08	32.60	-0.479	98.55%
q5	74.59	70.06	-4.536	93.92%
q6	6.35	6.39	0.040	100.63%
q7	81.19	82.44	1.245	101.53%
q8	87.31	85.16	-2.153	97.53%
q9	120.08	121.52	1.438	101.20%
q10	47.07	47.44	0.363	100.77%
q11	20.26	20.52	0.261	101.29%
q12	25.66	26.96	1.305	105.09%
q13	38.31	38.00	-0.308	99.20%
q14	18.43	21.40	2.969	116.11%
q15	32.87	31.78	-1.089	96.69%
q16	14.19	13.13	-1.060	92.53%
q17	103.96	104.52	0.563	100.54%
q18	148.00	147.93	-0.068	99.95%
q19	13.95	13.84	-0.115	99.18%
q20	29.59	29.65	0.058	100.20%
q21	260.57	265.38	4.803	101.84%
q22	12.47	12.52	0.053	100.42%
total	1268.56	1269.61	1.045	100.08%