Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The use case I'm referring to is when the client must be sharded to avoid CPU or network or disk bottlenecks.


Then that's exactly what consumer groups help with but it sounds like you want partitioning then - which is exactly what Kafka does but with a little more automation.

Run multiple Redis instances and use a simple hash based on whatever key you want to route messages and get the throughput you need. It's probably never going to be part of the core Redis logic but should be possible as a module to do the routing when used in a Redis cluster.


What I'm referring to is changing the partitioning dynamically by splitting streams, not just redis nodes. Here is one implementation example http://docs.aws.amazon.com/streams/latest/dev/kinesis-using-...

Doing it without server support is tricky.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: