Using PostgreSQL as a Dead Letter Queue for Event-Driven Systems
30 points
1 hour ago
| 2 comments
| diljitpr.net
| HN
▲
rbranson
37 minutes ago
[-]
Biggest thing to watch out with this approach is that you will inevitably have some failure or bug that will 10x, 100x, or 1000x the rate of dead messages and that will overload your DLQ database. You need a circuit breaker or rate limit on it.
reply
▲
shayonj
17 minutes ago
[-]
This! Only thing worse than your main queue backing off is you dropping items from going into the DLQ because it can’t stay up.
reply
▲
pletnes
15 minutes ago
[-]
If you can’t deliver to the DLQ, then what? Then you’re missing messages either way. Who cares if it’s down this way or the other?
reply
▲
RedShift1
4 minutes ago
[-]
The point is to not take the whole server down with it. Keeps the other applications working.
reply
▲
xyzzy_plugh
7 minutes ago
[-]
Not necessarily. If you can't deliver the message somewhere you don't ACK it, and the sender can choose what to do (retry, backoff, etc.)

Sure, it's unavailability of course, but it's not data loss.

reply
▲
rbranson
8 minutes ago
[-]
Sure, but you still need to design around this problem. It’s going to be a happy accident that everything turns out fine if you don’t.
reply
▲
exabrial
4 minutes ago
[-]
> FOR UPDATE SKIP LOCKED

Learned something new today. I knew what FOR UPDATE did, but somehow I've never RTFM'd hard enough to know about the SKIP LOCKED directive. Thats pretty cool.

reply