-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storage: better way to determine fetch_size
#505
Comments
My idea is that a fetch size vec corresponds to the fetch size for each column. if a certain size is 0, it will be skipped directly, and no io is needed. 🥺 Is that okay with you? @skyzh |
Could you please elaborate it in more details? Note that we must fetch equal size of rows for all columns. |
old -> [col1_fetch_size= 15, col2_fetch_size= 10, col3_fetch_size= 0] will use |
That's exactly what I want! Feel free to send a PR. If all of them are 0 (or None), we can use |
/assignme |
Currently, the logic is like:
risinglight/src/storage/secondary/rowset/rowset_iterator.rs
Lines 108 to 122 in fea5e0b
If there is any hint from column iterators, we choose the minimum of remaining items N, so that we can issue as few I/O as possible for this round. This number N can be:
ROWSET_MAX_OUTPUT
. This value will only be used when there's no hint from any of the columns. The naming of this const is somehow misleading 🤪We should find a better way to determine
fetch_size
. e.g., by removingfetch_hint
interface and solely base on the column index information.The text was updated successfully, but these errors were encountered: