Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Nimbus codebase to use the new nim-rocksdb API. #2054

Merged
merged 7 commits into from
Mar 5, 2024

Conversation

web3-developer
Copy link
Contributor

@web3-developer web3-developer commented Feb 29, 2024

The new version of nim-rocksdb makes the following breaking changes:

  • RocksDbInstance now named RocksDbReadWriteRef or RocksDbReadOnlyRef depending on readOnly mode.
  • contains renamed to keyExists.
  • del renamed to delete and no longer returns flag indicating if a record was deleted.
  • Backup features moved to BackupEngineRef type.
  • Low level c api moved to rocksdb/lib/librocksdb and is not exported by default.
  • C API types are updated to be objects so ptr prefix is required. This was required in order to update the header file wrapper to the latest version.
  • The RocksDbRef types no longer expose internal fields.

Due to these above changes a lot of files needed to be updated.

@web3-developer
Copy link
Contributor Author

While looking through the code I also discovered a memory leak due to calling allocCStringArray without a free. See docs for that function here: https://nim-lang.org/docs/system.html#allocCStringArray%2CopenArray%5Bstring%5D

I've pushed a fix so that we don't alloc a new cstring array in these cases.

rbl.db.store.db.rocksdb_ingest_external_file(
[rbl.filePath].allocCStringArray, 1,
rbl.db.store.cPtr.rocksdb_ingest_external_file(
cast[cstringArray](filePath.addr), 1,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the fix for the memory leak. Similar change in two other places.

return err(v.error)

ok(T(store: store))
let backupRes = openBackupEngine(backupsDir)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whenever possible, where there is no conflict between symbols, perhaps we can use valueOr, isOkOr, etc to improve code readability. I know there is still a lot of old code using isErr, well, they are old code.

example:

let backupRes = openBackupEngine(backupsDir).valueOr:
     return err(error)

# or
let backupRes = ? openBackupEngine(backupsDir)  # if both callee and caller have identical return type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yes looks much cleaner. I'll update.

@web3-developer web3-developer force-pushed the update-nim-rocksdb-breaking-changes branch from 9c40690 to 324a2da Compare March 5, 2024 03:44
@web3-developer web3-developer marked this pull request as ready for review March 5, 2024 03:45
@web3-developer web3-developer merged commit 11691c3 into master Mar 5, 2024
25 checks passed
@web3-developer web3-developer deleted the update-nim-rocksdb-breaking-changes branch March 5, 2024 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants