-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The change counter in the sqlite header isn't *always* updated #10
Comments
Oh wow, this is super interesting, thanks @pkhuong!
That's actually how it works right now! See https://github.com/jlongster/absurd-sql/blob/master/src/indexeddb/worker.js#L423 and the comment above it. Specifically https://github.com/jlongster/absurd-sql/blob/master/src/indexeddb/worker.js#L388 When a I don't think the garbage data is a problem -- as long as the filesize is correct and everything else in the file is in tact, I don't see how that would effect anything.
Flushing to disk in the middle of a transaction is something I want to understand more. Does it mainly happen in a huge write transaction where it can't hold all the data in memory? I think this should be fine because we grab the And now I understand your point about locks. Because we are already doing that, we sidestep this issue. A connection can never write in the middle of a transaction without already having a lock, which we have because we start it on Thanks for reporting this, it feel great to make sure we've covered the edge cases! |
Simplifying the locking protocol does seem to help make correctness more robust. However, I don't know if the list of conditions above is exhaustive. A dedicated out-of-band change counter might help you find additional issues. |
I think locking the file for a whole time that a I will keep this in mind though. It's definitely important we get this right, and I'm thinking about how to test it. I'm open to having our own change counter as well, but I want to have clear reasons to do so. I'll leave this open and keep thinking about it |
I see "Once we’ve grabbed a readwrite transaction, we can read some bytes from the file which represent SQLite’s “change counter”. If that counter is the same as what we had when we requested a write, it’s safe to write!" in the announcement post.
Unfortunately, I discovered that's only mostly true while working on https://github.com/backtrace-labs/verneuil-wip. Verneuil runs sqlite's tcl regressions tests with assertions enabled in its replicating VFS, and assertion failures in the bitwise replication check uncovered at least two scenarios where sqlite can change the data on disk without updating the header's version counter:
It probably makes sense for absurd-sql to manage its own dedicated change-tracking key-value entry, or maybe simplify the locking mechanism to jump over the "reserved/pending" states, and only have shared & exclusive.
The text was updated successfully, but these errors were encountered: