Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Slow scan with Spark runtime filter when split-preloading is turned on #6074

Open
zhztheplayer opened this issue Jun 13, 2024 · 0 comments
Labels
bug Something isn't working triage

Comments

@zhztheplayer
Copy link
Member

zhztheplayer commented Jun 13, 2024

In some cases when runtime filter is added, spark.gluten.sql.columnar.backend.velox.IOThreads=0 (which turns Velox's preload / prefetch off) could gain better performance than the default settings (spark.gluten.sql.columnar.backend.velox.IOThreads=<task slot num>).

Caused by facebookincubator/velox#10173

To temporarily bypass the issue if found, just set spark.gluten.sql.columnar.backend.velox.IOThreads=0.

@zhztheplayer zhztheplayer added bug Something isn't working triage labels Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant