Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Async processing is also a great way to sweep failures under the rug.

A lot of those failures won't even be detected, because you don't have proper monitoring, whereas in sync world you fail early and loudly.

And instead of "we had a downtime" you can sugarcoat it as "we had a processing delay". What's not to love?



As the article points out, I feel that messaging really shines when it comes to failures: Instead of an - at best - WARN or ERROR log line amongst millions of log lines, the failing message "pops out" of the messaging fabric, into a Dead Letter Queue (DLQ).

You shall obviously set up monitoring of the MQ and all its DLQs. Once you've done that, you will literally never have a failing processing flow that isn't caught. A successfully initiated Mats Flow will either run to completion, or it will DLQ. Guaranteed.

The MatsBrokerMonitor is a Mats-specific such monitor (compared to the generic monitor ActiveMQ comes with). The pages here are a bit lacking, but it at least explains a bit: https://mats3.io/docs/matsbrokermonitor/


The major problem with dead letter queues is that people naturally end up mixing recoverable and non-recoverable errors together, which results in all errors being ignored.

Proper monitoring typically requires application specific details and logic, which queues, being a dumb bus, typically don't have.

E.g. your client has a scheduled downtime every midnight, so you want your monitoring to not alert on messages produced by account X between 00:00 and 02:00. Oh, and ideally these exclusion rules are set up by non-technical folks (maybe even by customer himself).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: