You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The recently open-sourced yandex-db database has a feature enabling users to export a database to both the file system and any S3-compatible storage. The import story is similarly easy.
It would be great to have the same feature in couchdb, specifically to help with the replication of very large databases (hundreds of gigabytes, dozens of terabytes or even petabytes) to a different data center. Transmitting everything over the wire is impractical in these situations. In the 1.X era, one simply needed to take a snapshot of the .couch file, but with clustering there is no clear solution to this problem.
Desired Behaviour
Users should be able to both export and import and a database in one click from Fauxton or using curl. The feature only needs to support the export of a whole database (no need to support partial exports), and the import of the whole database into a new database. Supporting imports to an existing database would unnecessarily introduce the problem of conflict handling. For simplicity, users can simply import to a new database and realize filtered replications as needed to a local database.
Additional context
Sometimes it is needed to move or replicate very large databases to a different data center not necessarily having a 10gbps internet link. This feature would help facilitate the transfer of the bulk of data, while newest data will be able to be transferred using the replication protocol.
The text was updated successfully, but these errors were encountered:
Summary
The recently open-sourced yandex-db database has a feature enabling users to export a database to both the file system and any S3-compatible storage. The import story is similarly easy.
It would be great to have the same feature in couchdb, specifically to help with the replication of very large databases (hundreds of gigabytes, dozens of terabytes or even petabytes) to a different data center. Transmitting everything over the wire is impractical in these situations. In the 1.X era, one simply needed to take a snapshot of the .couch file, but with clustering there is no clear solution to this problem.
Desired Behaviour
Users should be able to both export and import and a database in one click from Fauxton or using curl. The feature only needs to support the export of a whole database (no need to support partial exports), and the import of the whole database into a new database. Supporting imports to an existing database would unnecessarily introduce the problem of conflict handling. For simplicity, users can simply import to a new database and realize filtered replications as needed to a local database.
Additional context
Sometimes it is needed to move or replicate very large databases to a different data center not necessarily having a 10gbps internet link. This feature would help facilitate the transfer of the bulk of data, while newest data will be able to be transferred using the replication protocol.
The text was updated successfully, but these errors were encountered: