Avoiding duplicate objects in Django querysets
25 points
4 days ago
| 3 comments
| johnnymetz.com
| HN
jiaaro
2 minutes ago
[-]
Exists is a useful tool that you should certainly know how to use. Whether or not it's faster than distinct depends on the rest of the query.

Also, some databases (like clickhouse) allow for `any` joins which avoid producing duplicate rows. For example:

    select author.*
    from author
    left any join book on (book.author_id = author.id and book.title like 'Book%')
reply
augusteo
2 hours ago
[-]
Nice writeup. The Exists subquery approach is definitely the cleanest.

One thing worth mentioning: if you're hitting this problem frequently, it might be worth reconsidering the query patterns themselves. We had a similar issue at work where we kept adding `.distinct()` everywhere, and eventually realized we were doing the filtering wrong upstream.

The PostgreSQL-specific `distinct(*fields)` with the ORDER BY restriction is one of those things that trips people up. The error message isn't great either. "SELECT DISTINCT ON expressions must match initial ORDER BY expressions" is technically correct but doesn't explain why or what to do about it.

Good call recommending Exists as the default approach. It's more explicit about intent too.

reply
ducdetronquito
2 hours ago
[-]
Good read, TIL!

That being said, I use Django daily for 10 years but I don’t understand the ORM besides basic CRUD. Even a simple group by looks weird.

Writing plain SQL feels easier and more maintainable in the long run.

reply