Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Also on multi-node failures smaller partitions actually reduce the amount of critical data to re-replicate to keep the probability low and even give you ability to prioritize that.

For example losing two nodes with 100 partitions on each might give you only like 5 partitions that happen to have replicas living on both nodes and that need to be re-replicated as soon as possible from a single still existing replica, but the rest of the replicas for 190 different partitions have 2 existing replicas and can wait til those 5 high-priority ones are re-replicated.

I guess this answers Martin's question - "you can choose between a high probability of losing a small amount of data, and a low probability of losing a large amount of data! Is the latter better?" - no, the latter is worse, much-much worse.



Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: