pq is a simpler approach: It's a Postgres-backed background-task library for Python, using SELECT ... FOR UPDATE SKIP LOCKED for concurrency safety. You just run your N task workers, that's it. The stuff I think is worth knowing about: - Transactional enqueueing -- you can enqueue a task inside the same DB transaction as your writes. If the transaction rolls back, the task never exists. This is the thing Redis literally can't give you. Fork isolation -- every task runs in a forked child process. If it OOMs, segfaults, or leaks memory, the worker just keeps going. The parent monitors via os.wait4() and handles timeouts, OOM kills, and crashes cleanly. - Periodic tasks -- intervals or cron expressions, with overlap control (skip the tick if the previous run is still going), pause/resume without deleting the schedule. Priority queues -- five levels from BATCH to CRITICAL. You can dedicate workers to only process tasks with specific priorities. - Three tables in your main database schema: pq_tasks for one-off tasks, pq_periodic for periodic tasks and pq_alembic_version to track its own schema version (pq manages its own migrations).
There's also client IDs for idempotency and correlation with other tables in your application DB, upsert for debouncing (only the latest version of a task runs), lifecycle hooks that execute in the forked child (useful for fork-unsafe stuff like OpenTelemetry), and async task support.
What it won't do: high throughput (you're polling Postgres). If you need 10k+ tasks/sec or complex DAGs, use something else. For the kind of workload most apps actually have, it's probably sufficient.
pip install python-pq // uv add python-rq repo at https://github.com/ricwo/pq, docs at https://ricwo.github.io/pq