Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

How does it deal with events like disk failure, network partitions, and concurrent updates? The design documentation is rather light, so it's really hard to make out how this actually distributes data.

You say it doesn't have any concept of eventual consistency. Yet how does it coordinate updates to nodes? Does it do two phase commit? Paxos?



Every node in both hash rings is replicated using master-master replication - i.e. node A and node A' can both receive updates and reads. Node A and node A' sync their updates via an update log and can fail at any time and come back at any time without taking down the system.

Additionally, if high availability is really a big issue, then a node A''' can be introduced that can be in another data center.

If you add nodes to the storage ring, then some of the existing keys will be invalidated. To solve this issue and the issue of routing a lookup ring is created. Lookup ring holds a pair (key, storage_ring_location). The system will automatically update (key, storage_ring_location) if it's at some point invalidated (such as that key does not point to node A, but node D).

I have tried to find an easy solution for a rather complex problem. Keeping membership state, doing Paxos and keeping routing tables would have been much more time consuming to make - so I have tried to solve the problem from another angel (by using master-master replication for high availability).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: