cache.add() as an atomic lockDjango's cache framework (from django.core.cache import cache) is a backend-agnostic key/value store. Three core methods look similar but behave very differently under concurrent access:
cache.set(key, value) — always writes. No race protection.cache.get(key) — reads. Two callers reading at once both see the same state.cache.add(key, value, timeout=...) — atomic set-if-absent. Returns True if the key was new, False if it already existed. The atomicity is the whole point.That makes cache.add a one-line distributed lock with a built-in TTL. On Redis it compiles to SET key value NX EX <timeout>.
The classic mistake is to write if not cache.get(key): cache.set(key, ...) — two workers can both pass the get before either sets, and now both think they hold the lock. cache.add collapses both steps into one atomic operation that the database (or Redis) arbitrates.
The per-host politeness floor in crawl_feeds:
lock_key = f"crawl:host:{host}"
if not cache.add(lock_key, 1, timeout=1800):
# another fetch claimed this host in the last 30 min — defer
UserFeed.objects.filter(pk=feed.pk).update(
next_check_at=now() + HOST_POLITENESS_INTERVAL,
)
continue
Two Celery workers crawling at the same time would each call cache.add; exactly one gets True, the other gets False, and we never double-fetch a host. No explicit lock object needed.
add — rewrite the lock as cache.get/cache.set and write a test that simulates two workers (just two consecutive calls in one thread is enough — get returns None for both). Confirm both pass the check.
Done when: you have a test that proves get+set is racy and cache.add isn't.