Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to export a database to a file and to any S3 compatible storage #4043

Open
nkosi23 opened this issue May 29, 2022 · 0 comments

Comments

@nkosi23
Copy link

nkosi23 commented May 29, 2022

Summary

The recently open-sourced yandex-db database has a feature enabling users to export a database to both the file system and any S3-compatible storage. The import story is similarly easy.

It would be great to have the same feature in couchdb, specifically to help with the replication of very large databases (hundreds of gigabytes, dozens of terabytes or even petabytes) to a different data center. Transmitting everything over the wire is impractical in these situations. In the 1.X era, one simply needed to take a snapshot of the .couch file, but with clustering there is no clear solution to this problem.

Desired Behaviour

Users should be able to both export and import and a database in one click from Fauxton or using curl. The feature only needs to support the export of a whole database (no need to support partial exports), and the import of the whole database into a new database. Supporting imports to an existing database would unnecessarily introduce the problem of conflict handling. For simplicity, users can simply import to a new database and realize filtered replications as needed to a local database.

Additional context

Sometimes it is needed to move or replicate very large databases to a different data center not necessarily having a 10gbps internet link. This feature would help facilitate the transfer of the bulk of data, while newest data will be able to be transferred using the replication protocol.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant