Why can a service leak memory even when reference counting works?
Python 3.10+; standard library only. GIL discussion refers to conventional CPython builds, not optional free-threaded builds.
Detailed answer
A leak can be reachable data that the application no longer needs: unbounded caches, retained request objects, callback registrations and global collections are common causes.
Reference counting releases unreachable objects without cycles; cyclic garbage collection handles many unreachable cycles. Neither collector can discard an object still strongly reachable from a live cache.
Use tracemalloc snapshots around a reproducible workload, compare allocation sites and distinguish Python heap growth from native allocation or allocator fragmentation. Set cache limits and remove registrations.
Internal working
A root reference keeps an entire object graph alive. Weak references allow observers without owning the target, but can disappear between reads and must be checked.
Practical scenario
A metrics registry stores a closure for every request; each closure retains the request and its payload. Remove registrations or store bounded aggregate metrics.
import tracemalloc
tracemalloc.start()
values = [str(i) for i in range(1000)]
print(tracemalloc.get_traced_memory())
del values
tracemalloc.stop()Common mistakes
- Assuming gc.collect fixes reachable caches.
- Confusing RSS with live Python object size.
Key takeaways
- Inspect reachability.
- Measure allocation deltas.
- Bound retention by design.
Follow-up challenges
- How do weak references affect ownership?
- How would you find native memory growth?