-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sharding #50
Comments
yes, that is how i would do it. read the dynamo paper, it describes that architecture in detail - it's also essentially how bit torrent works from a big picture perspective |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Dominic, first of all thank you for the amazing library, which we successfully could integrate to our project, using streams on webrtc.
We are trying to implement a system where every browser becomes a node, eg. 1000 browsers, every 10 of them become a cluster, every peer in the cluster holds the same data (1/100th of the total), thus 100 clusters contain the full dataset. This is a network of browsers, therefore, each of them can appear/disappear at random (this example assumes not all 10 at the same cluster would disappear at once). We want to make sure when a browser joins the network, it holds only 100th of the dataset (numbers are figurative) so that they don't have to download/keep a lot of data. Peers in the cluster replicate, clusters will do the sharding.
Maybe a consistent hashring is the way to go, maybe finger table, I was wondering if you have given this a thought, if so, what would you advise?
The text was updated successfully, but these errors were encountered: