> I don't understand how you can make queueing high-performance. Queueing things is literally the opposite of making things fast.
You likely need to read about message queues, why they are used, and why they have performance constraints. Message queues are a system, and the system to maintain those queues and content has an overhead. The project is building a solution that is higher performing than comparable systems.
Queuing things often makes things faster (!) as you are usually dealing with limited resources (ex: web and job workers) and need to operate on other systems (ex: payments) that have their own distinct capacity.
If you have 1000 web workers, and your upstream payment provider supports 10 payments per minute, you'll quickly have timeouts and other issues with 1000 web workers trying to process payments. Your site won't process anything else while waiting for the payment provider if all web workers are doing this task.
A queue will let you process 1000 web requests, and trickle 10 requests to your payment provider. At some point that queue will fill, but that's a separate issue.
Meanwhile, your 1000 web workers are free to process the next requests (ex: your homepage news feed).
A queue is a thing to be managed carefully when designing systems:
- you need to ensure that your producers don't produce at a faster rate than your consumers can keep up
- if the consumer is unable to handle the request now, queueing it and handling it later is often not the right thing to do. Most likely by the time the resource is available the nature of the request will change.
TCP itself is already a queue. All these message queue systems also make the silly decision of layering their own queueing on top of TCP, leading to all sorts of problems.
Basically those things only sort of work if you have few messages and are happy with millisecond latency.
It really depends what you design your system to handle. A sudden burst of traffic that can be spread for minutes is fine (ex: Elasticsearch indexing requests can usually be delayed through background jobs).
> Basically those things only sort of work if you have few messages and are happy with millisecond latency.
Not really... queues are great to defer processing or have longer running job than your HTTP and TCP timeouts will allow. Building large data export won't happen within a single HTTP request.
It's a bit difficult to cover as it is highly dependent on the queue system you use.
You'd usually want your queue system to fail the enqueue if it is full, and you'd want monitoring to ensure the queue isn't growing at an unsustainable rate.
It also forces you to think a bit about your message payload (rich data or foreign keys the worker loads).
RabbitMQ, Redis-based queues (ex: ActiveJob or sidekiq), Gearman, and others will all offer different mechanisms to tackle full queues.
Queueing things can result in better throughput because you don't have to wait for the other side to process your last message before you can begin work on the next one.
That's just buffering. A lot of people use Kafka this way, as an impedance adapter for a producer with an internal architecture that can't tolerate blocking on production. Of course this requires you to unrealistically assume that Kafka is a zero-impedance message sink.
But what I think the other post is alluding to is the fact that in-order delivery is often not a requirement, and can be an anti-feature. I know that in every use of Kafka I have personally encountered the relative order of messages was quite irrelevant but the in-order implementation of Kafka converts the inability to process a single record into an emergency.
You’re misunderstanding the discussion. If you think of queuing as “waiting to process” then yes it’s slower than not waiting. But that’s not at all the necessary implication of these components. In fact “synchronous” requests are buffered (queued?) internally and can be delayed before processing. So the distinction is irrelevant to performance. And the implementation is what matters.
UDP doesn't provide reliable delivery, so you would need to implement some kind of delivery assurance in your application.
UDP also has very small message (datagram) size limits, so you would also need to implement some way to fragment and re-combine messages in your application.
At this point you've built an ad-hoc re-implementation of 80% of TCP.
i'm not an expert here but each message is pretty important right? What happens if you put a UDP datagram on the wire and it never shows up? Wouldn't you have to come up with some kind of signal to indicate the message made it to the receiver? After that you'll probably start signaling delivery of groups of messages instead of individually and then ordering them on the receiver by when they were sent and then maybe some validation of integrity and sender backoff mechanism for network congestion... and then you've made TCP.
TCP means that all bytes must be copied to a local buffer in order to be re-sent in the case the other end didn't receive them (even though by that time they're probably old and irrelevant). Once the buffer is full you cannot send more data and must wait for the other end to ack them (potentially indefinitely, though most implementations will eventually time out after long enough). This means that if you wanted to send 10Gbps unthrottled to another continent, you'd need a buffer of 500MB per TCP session, assuming no data loss, and consumer keeping up at line rate.
TCP is just not a good fit for anything that needs high-performance or bounded sending times. What it's good for is giving you a simple high-level abstraction for streaming bytes when you'd rather not design your networked application around networking.
Building the right logic for your application is not difficult (most often, just let the requester resend, which you needs to do anyway in the age of fault-tolerant REST services, or just wait for more recent data to be pushed) and easier than tweaking TCP to have the right performance characteristics.
