Memory Management and Garbage Collection in Python

In languages like C, you're responsible for memory yourself — forget to free something and you leak it; free it twice and you crash. python memory management removes both risks by design. This article covers garbage collection python actually relies on: reference counting as the primary mechanism, why circular references defeat it on their own, and the generational garbage collector that exists specifically to catch what reference counting can't.

Python Handles Memory So You (Usually) Don't Have To

The contrast with manual memory management

In a language like C, you explicitly allocate memory and are responsible for explicitly freeing it later. Forget to free something, and that memory leaks — it stays reserved forever, even though nothing in your program can still use it. Free the same memory twice, or use it after freeing it, and you risk a genuine crash, or worse, silent memory corruption. Python's automatic memory management removes both of these entire categories of bugs by design — you never manually allocate or free memory yourself at all.

Two mechanisms working together

Python actually uses two mechanisms together, not one: reference counting is the primary, immediate mechanism, handling the overwhelming majority of everyday memory cleanup instantly and predictably. A generational garbage collector exists as a backup system, specifically for a category of case reference counting genuinely cannot handle on its own — circular references, covered in Section 3.

What this article covers

How references and reference counting actually work, why circular references specifically break pure reference counting, how the generational garbage collector solves that gap, and honest, practical guidance on when — rarely — you'd actually want to interact with the gc module directly.

Reference Counting: The Primary Mechanism

The mental model: variables are labels, not boxes

This is worth internalizing clearly, since it underlies everything else in this article: Python variables are labels (or names) pointing to objects in memory — not boxes that directly contain values themselves. Multiple different variable names can point to the very same object simultaneously.

a = [1, 2, 3]
b = a   # 'b' is a second label pointing to the SAME list object, not a copy

b.append(4)
print(a)   # [1, 2, 3, 4] — a sees the change too, since they're the same object
Every object carries a reference count

Every object in Python maintains an internal count of exactly how many references currently point to it. Several everyday actions increase that count: assigning it to a variable, passing it as a function argument, or storing it inside a container like a list or dictionary. Other actions — del, or reassigning a variable to point elsewhere — decrease it.

import sys

a = [1, 2, 3]
print(sys.getrefcount(a))   # includes the extra reference from getrefcount's own argument

b = a   # a second reference to the same object
print(sys.getrefcount(a))   # count increased

del b   # one fewer reference
print(sys.getrefcount(a))   # count decreased again

(sys.getrefcount()'s own reported number is always one higher than you might expect, since passing the object into getrefcount() itself temporarily creates one more reference — worth knowing if you experiment with this directly.)

When the count hits zero

The moment an object's reference count reaches zero — meaning nothing in the program can reach it anymore — CPython reclaims that memory immediately, right then, with no waiting and no separate collection pass required. This is precisely why reference counting alone is fast and highly predictable for the overwhelming majority of objects: cleanup happens the instant it's genuinely safe, rather than being deferred to some later, separate garbage-collection cycle.

class Resource:
    def __del__(self):
        print("Resource cleaned up")

r = Resource()
del r
# Resource cleaned up — printed immediately, the moment the reference count hit zero

Circular References: Where Reference Counting Falls Short

The problem

Reference counting has one genuine, fundamental blind spot: if two or more objects reference each other in a loop, their reference counts can never naturally reach zero on their own — even after every external variable pointing to them has been deleted, they still each hold a reference to the other, keeping each other's count above zero forever.

A concrete example
class Node:
    def __init__(self, name):
        self.name = name
        self.partner = None

a = Node("A")
b = Node("B")

a.partner = b   # a references b
b.partner = a   # b references a — a circular reference

del a
del b

After both del a and del b, there are no longer any variables in your program that can reach either Node object directly. But each object still holds a reference to the other, inside its own .partner attribute — a's object still has a nonzero reference count (from b's .partner), and vice versa. Pure reference counting alone would leave both of these objects sitting in memory forever, completely unreachable from your actual program, but never cleaned up.

Why this matters practically

Circular references are a genuinely common, often-overlooked source of real memory leaks in production applications — particularly with tree- or graph-like data structures (a node referencing its parent, which references its children, which reference their parent again) or parent-child object relationships generally, where it's natural and often necessary for two related objects to hold references back to each other.

The Generational Garbage Collector

How it solves the cycle problem

Python's generational garbage collector exists specifically to catch exactly this case. Periodically, it scans a subset of objects, effectively building a reference graph, and identifies groups of objects that reference each other in a cycle but aren't reachable from anything else in the active program — exactly the Node example above. Once such an unreachable cycle is identified, the collector reclaims the entire group together, something reference counting alone structurally cannot do.

The three generations

The collector organizes tracked objects into three "generations," based on how long they've survived so far:

  • Generation 0 (young) — newly created objects. Scanned most frequently, since the overwhelming majority of objects in a typical Python program are short-lived (temporary variables, intermediate calculation results) and get cleaned up quickly, whether through reference counting or an early collection pass.

  • Generation 1 and Generation 2 (older) — objects that have survived one or more collection passes already, and get promoted to an older generation, which is scanned considerably less frequently.

This generational structure exists purely for efficiency: scanning every object in the entire program on every single collection pass would be genuinely wasteful, since most objects that are going to become garbage do so quickly, while a smaller number of longer-lived objects (module-level data, long-running caches) rarely need to be rescanned at all once they've proven themselves to be stable and long-lived.

Interacting with the gc module
import gc

# Force an immediate, full collection
gc.collect()

# Inspect the current collection thresholds (generation 0, 1, 2)
print(gc.get_threshold())   # e.g., (700, 10, 10)

# Inspect current object counts per generation
print(gc.get_count())

# Temporarily disable automatic collection
gc.disable()

# Re-enable it
gc.enable()

gc.get_threshold() shows the number of allocations (relative to deallocations) that trigger a collection pass for each generation — the collector doesn't run on a fixed timer; it runs based on allocation activity crossing these thresholds. gc.collect() forces an immediate, full collection pass across all generations, regardless of whether the normal thresholds have actually been reached.

Practical Guidance: Weak References and When to Intervene

weakref for references that don't keep an object alive

weakref.ref() (and the related weakref.finalize()) let you refer to an object without increasing its reference count at all — genuinely useful in caching scenarios, where you specifically don't want a cache entry to be the only thing keeping an otherwise-unused object alive in memory.

import weakref

class LargeObject:
    def __init__(self, name):
        self.name = name

obj = LargeObject("data")
weak_ref = weakref.ref(obj)

print(weak_ref())   # <__main__.LargeObject object at 0x...> — still accessible

del obj
print(weak_ref())   # None — the object was actually freed, since the weak reference didn't keep it alive

Because weak_ref doesn't count toward obj's reference count, deleting the only real reference (obj itself) allows the object to be genuinely cleaned up immediately, exactly as it would without the weak reference existing at all — weak_ref() correctly returns None afterward, rather than pointing to a now-dangling object.

Honest guidance on manual intervention

This is worth stating plainly: the overwhelming majority of applications never need to call gc.collect() manually. Python's automatic reference counting plus generational collection handles the vast majority of real-world memory management correctly and efficiently, entirely on its own, without any intervention from you.

Reach for manual gc.collect() calls selectively, in specific, genuinely justified situations — memory-intensive batch jobs where you know a large number of circular structures were just torn down and want to reclaim that memory immediately rather than waiting for the next automatic pass, or immediately before a memory-sensitive operation where you want to start with as clean a memory footprint as reasonably possible. It is not something to sprinkle in as a routine habit "just in case" — forcing frequent manual collection adds real, measurable overhead (scanning a meaningful portion of your program's objects isn't free), typically without any corresponding benefit, since the automatic system is already handling things correctly in the background.

Best practices for avoiding leaks in the first place

Prefer context managers (with) for deterministic cleanup of files, sockets, and locks — as covered extensively in the earlier with statement and contextlib articles, this guarantees cleanup happens at a precise, predictable moment, rather than depending on garbage collection timing (which, for ordinary objects without circular references, is generally immediate anyway via reference counting — but with makes that guarantee explicit and doesn't rely on it at all).

Avoid unnecessary circular references in your own custom classes. Sometimes a genuine circular structure (a tree with parent-pointing children) is the correct, natural design — but it's worth being deliberate about it, since every circular reference is one more thing relying specifically on the generational collector (rather than immediate reference counting) to eventually clean up.

Monitor memory growth in long-running services, rather than guessing at whether something is leaking. gc.get_objects() gives you a snapshot of every object currently tracked by the collector, which you can compare across two points in time to spot unexpected, sustained growth. For a broader, more practical view of your program's actual memory footprint over time, a tool like psutil (a third-party package for querying system and process-level resource usage) is generally more directly useful than manually inspecting gc internals — genuinely worth reaching for if you suspect a long-running service's memory usage is growing in a way that shouldn't be happening, rather than trying to diagnose it purely by intuition.

PREVIOUSNEXT LESSON