A first look at Django's new background tasks
66 points
by roam
5 hours ago
| 6 comments
| roam.be
| HN
JakaJancar
57 minutes ago
[-]
Assuming you're fine with keeping the queue in postgres, I've used Procrastinate and it's great:

https://procrastinate.readthedocs.io/en/stable/index.html

Core is not Django-specific, but it has an optional integration. Sync and async, retries/cancellation/etc., very extensible, and IMO super clean architecture and well tested.

IIRC think the codebase is like one-tenth that of Celery.

reply
TkTech
32 minutes ago
[-]
If you like Procastinate, you might like my Chancy, which is also built on postgres but with a goal of the most common bells and whistles being included.

Rate limiting, global uniqueness, timeouts, memory limits, mix asyncio/processes/threads/sub-interpreters in the same worker, workflows, cron jobs, dashboard, metrics, django integrations, repriotization, triggers, pruning, Windows support, queue tagging (ex: run this queue on all machines running windows with a GPU, run this one on workers with py3.14 and this one on workers with py3.11) etc etc...

https://tkte.ch/chancy/ & https://github.com/tktech/chancy

The pending v0.26 includes stabilizing of the HTTP API, dashboard improvements, workflow performance improvements for workflows with thousands of steps and django-tasks integration.

reply
matsemann
4 hours ago
[-]
How is the typing support? We just had downtime because a change to a celery task didn't trigger mypy to complain for all call sites until runtime. Too many python decorators aren't written with pretty weak typing support.
reply
roam
3 hours ago
[-]
With regards to args and kwargs? None. Your callable is basically replaced with a Task instance that’s not callable. You need to invoke its enqueue(*args, **kwargs) method and yeah… that’s of course not typed.
reply
halfcat
2 hours ago
[-]
Static analysis will never be fully robust in Python. As a simple example, you can define a function that only exists at runtime, so even in principle it wouldn’t be possible to type check that statically, or even know what the call path of the functions is, without actually running the code in trace/profiler mode.

You probably want something like pydantic’s @validate_call decorator.

reply
tomjakubowski
1 hour ago
[-]
> you can define a function that only exists at runtime, so even in principle it wouldn’t be possible to type check that statically

Can you say more, maybe with with an example, about a function which can't be typed? Are you talking about generating bytecode at runtime, defining functions with lambda expressions, or something else?

reply
ethagnawl
1 hour ago
[-]
This is an exciting development. I imagine I'll continue using Celery in most cases but being able to transparently swap back-ends for testing, CI, etc. is very compelling.

I haven't looked into this in any detail but I wonder if the API or defaults will shave off some of the rough edges in Celery, like early ACKs and retries.

reply
vforgione
2 hours ago
[-]
I’ve been using the django-tasks library in production for about a year. The database backend and simple interface have been great. It definitely isn’t intended to replace all of celery, but for a simple task queue that doesn’t require additional infrastructure it works quite well.
reply
the__alchemist
3 hours ago
[-]
This is great! The prev recommendation was usually a lib called celery that I wasn't able to get working. I don't remember the details, but it had high friction points or compatibility barriers I wasn't able to overcome. This integration fits Django's batteries included approach.

I've been handling this, so far, with separate standalone scripts that hook into Django's models and ORM. You have to use certain incantations in an explicit order at the top of the module to make this happen.

reply
halfcat
2 hours ago
[-]
> scripts that hook into Django's models and ORM

Django has management commands for that [1].

When you use Django over time, you experience this pleasant surprise over and over when you need something, “oh, Django already has that”

[1] https://docs.djangoproject.com/en/5.2/howto/custom-managemen...

reply
fud101
1 hour ago
[-]
Django this is about 10 years too late. It's frustrating because we use all manner of hacks to work around this being part of the builtin story.
reply