Understanding Packages and init.py

The previous article introduced packages briefly, as the natural next step once a project outgrows a handful of loose modules. This article covers python packages and init.py explained in real depth — what __init__.py actually does, the genuinely important distinction between regular and namespace packages, and how to use __init__.py deliberately to shape a clean, well-organized public API.

Introduction: From Modules to Packages

As a quick recap tying back to the previous article: a module is a single .py file. A package is a directory that bundles multiple related modules together into a single, organized, importable unit.

mypackage/
├── __init__.py
├── math_operations.py
└── string_operations.py

A package in Python is fundamentally a collection of modules that can be imported into other Python files as one cohesive unit, rather than needing to import each individual module file separately and manually track how they relate to each other.

What this article covers

Exactly what __init__.py does when it's present, why it's often optional today (a real, meaningful shift from Python's older behavior), and how to use it deliberately and well once your project genuinely benefits from it.

What init.py Actually Does

Historically required, now often optional

For a long stretch of Python's history, __init__.py was strictly required — its mere presence (even completely empty) was what told Python "this directory is a package, not just an ordinary folder." Since Python 3.3, thanks to PEP 420, this is no longer strictly true — __init__.py is now often optional for basic importing to work at all, through a mechanism called namespace packages, covered fully in Section 3.

What happens when init.py IS present

When a package does include an __init__.py file, the code inside it runs exactly once, automatically, the very first time that package is imported anywhere in your program — regardless of how many times, or from how many different files, you subsequently import from it afterward.

A simple demonstration

# mypackage/__init__.py
print("mypackage is being initialized")

# main.py
import mypackage
import mypackage   # importing it again

from mypackage import something

mypackage is being initialized

That message prints exactly once, even though mypackage is referenced three separate times across those lines. This connects directly to the module caching behavior covered in the earlier imports article: once a package has been imported and initialized, Python caches it in sys.modules, and every subsequent import — anywhere in your program, from any file — reuses that already-initialized version rather than re-running __init__.py's code again. This makes __init__.py a genuinely natural place for lightweight, one-time setup that a package's contents might depend on.

Regular Packages vs. Namespace Packages

Namespace packages: no init.py required

A directory that contains .py files but no __init__.py file can still function as an importable package, thanks to PEP 420 — Python calls this a namespace package, and it behaves slightly differently from a traditional, "regular" package.

# In a directory with no __init__.py at all
import mypackage   # this can still work, as a namespace package

A practical difference worth knowing

Beyond simply not requiring __init__.py, namespace packages have one genuinely distinctive property: a namespace package's __path__ isn't tied to a single physical directory the way a regular package's is. Multiple separate directories, potentially in entirely different locations on your file system, can contribute modules to the same logical namespace package, and Python will merge them together at import time.

/location_a/mypackage/module_a.py
/location_b/mypackage/module_b.py

If both /location_a and /location_b are somewhere on sys.path, and neither mypackage directory contains an __init__.py, Python can treat both as contributing to a single, unified mypackage namespace — letting you do from mypackage import module_a and from mypackage import module_b as though they'd always lived in the same directory together, even though they physically don't. This is a genuinely useful capability for certain advanced use cases — notably, allowing a large ecosystem of plugins to all contribute to a shared namespace without needing to coordinate directory structure ahead of time — but it's also a real source of confusion if you don't know it exists, since two entirely unrelated directories can silently merge together under the same package name if you're not deliberate about it.

There's also a minor, generally negligible performance consideration: namespace packages can be marginally slower to import than regular packages, since Python's import machinery has to do additional work checking for and potentially merging contributions from multiple locations, rather than resolving directly to a single, known directory.

Why most everyday projects still explicitly include init.py

Given all of this, most everyday projects continue to include an explicit __init__.py (even if it's completely empty) rather than relying on the implicit namespace-package behavior — for two genuinely practical reasons: clarity (an explicit __init__.py unambiguously signals "this is a deliberate, self-contained package," to both Python and to anyone reading your project's structure) and avoiding accidental namespace merging (an empty __init__.py explicitly marks a package as a regular package, which prevents the "multiple directories silently combining" behavior from happening by accident, if you didn't actually intend it).

Using init.py to Shape a Clean Public API

Re-exporting names from deeper submodules

One of the most genuinely useful things __init__.py can do is re-export names from deeper submodules up to the package root, letting users of your package write shorter, cleaner import statements:

# mypackage/server.py
def start_server():
    print("Server starting...")

# mypackage/__init__.py
from .server import start_server

# Without the re-export, users would need:
from mypackage.server import start_server

# With the re-export in __init__.py, users can instead write:
from mypackage import start_server

Both ultimately work, but the second is noticeably cleaner — and it also means you're free to reorganize your package's internal file structure later (splitting server.py into several smaller files, say) without breaking anyone's existing from mypackage import start_server import, since the re-export in __init__.py continues to expose that same name at the same top-level location, regardless of exactly where the underlying implementation actually lives internally.

Controlling from package import * with all

Defining __all__ — a list of strings — in __init__.py explicitly controls exactly what gets imported when someone runs the wildcard from package import * (covered as generally discouraged in the earlier imports article, but still worth controlling precisely when it does happen):

# mypackage/__init__.py
from .server import start_server, stop_server
from .server import _internal_helper   # a private-by-convention helper

__all__ = ["start_server", "stop_server"]   # explicitly excludes _internal_helper

from mypackage import *

start_server()   # works — listed in __all__
_internal_helper()
# NameError: name '_internal_helper' is not defined — correctly excluded

This gives you precise control over your package's public surface, keeping internal helper functions and implementation details out of a user's namespace even if they do use a wildcard import — genuinely useful for larger, published packages where you want a clear, deliberate boundary between "this is part of the supported public API" and "this is an internal implementation detail that could change without notice."

Package-level metadata

__init__.py is also the conventional home for simple, informational package-level metadata:

# mypackage/__init__.py
__versi "1.2.0"
__author__ = "Alex Chen"

import mypackage
print(mypackage.__version__)   # 1.2.0

This metadata is purely informational — it doesn't affect how the package actually functions — but it's a widely followed convention, genuinely useful for anyone (including tooling) that needs to programmatically check which version of a package is currently installed.

Best Practices for Real Projects

The core rule: keep init.py thin

This is worth treating as close to a hard rule: keep __init__.py limited to lightweight setup and re-exports only — never heavy logic like establishing a database connection, loading a large machine learning model, or performing expensive computation. Remember, from Section 2, that this code runs automatically on every single import of the package, anywhere in your program — heavy logic sitting in __init__.py means every part of your codebase that imports even a small, specific piece of your package pays that same expensive cost, whether or not it actually needed whatever that heavy setup was for.

# DON'T do this in __init__.py
import some_expensive_ml_library
model = some_expensive_ml_library.load_model("huge_model.bin")   # runs on EVERY import!

# DO this instead — load it lazily, only when actually needed
def get_model():
    import some_expensive_ml_library
    return some_expensive_ml_library.load_model("huge_model.bin")

Structuring subpackages

For a package containing its own nested subpackages, each subdirectory intended as a package needs its own __init__.py (or is deliberately left out, following the namespace-package pattern from Section 3), applied consistently at every level of nesting:

mypackage/
├── __init__.py
├── core/
│   ├── __init__.py
│   └── engine.py
└── utils/
    ├── __init__.py
    └── helpers.py

Each __init__.py in this structure follows exactly the same principles covered throughout this article — potentially re-exporting names from its own submodules, kept thin, and existing at every level where you want that level to be explicitly treated as its own regular package.

A modern pattern: lazy imports

For large libraries with many submodules, where importing the entire package eagerly would be genuinely slow (each submodule potentially pulling in its own heavy dependencies), a modern pattern uses a module-level __getattr__ function (available since Python 3.7, via PEP 562) inside __init__.py to defer a submodule's actual loading until a user genuinely accesses it:

# mypackage/__init__.py
def __getattr__(name):
    if name == "heavy_module":
        from . import heavy_module
        return heavy_module
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

import mypackage
# heavy_module hasn't actually been imported yet at this point

mypackage.heavy_module.do_something()
# NOW heavy_module is actually imported, lazily, at the moment it's first accessed

This keeps a large package's initial import fast, even if it contains many submodules with expensive dependencies, since each one only gets loaded the moment something actually asks for it — a genuinely useful pattern once your own package grows large enough that eager, upfront loading of everything becomes a real, measurable startup-time cost.