Python File Structure

From Files to Organization

As a Python project grows, related code needs an organized structure. Python packages provide that structure: a package is a folder containing Python modules and a special __init__.py file that marks the folder as a package. Packages can also contain subpackages, allowing a project to be arranged across multiple levels.

The Containment Hierarchy

A package is not just a collection of unrelated files. It provides a level of containment above individual modules. A module belongs inside a package, and a subpackage is itself a package inside another package. This creates a hierarchy that can group related modules together and then group those groups within a larger project structure.

In this generated arrangement, project_package is the outer package. reports and users are subpackages within it. The files monthly.py, annual.py, and profiles.py represent modules grouped according to the subpackage that contains them. The diagram illustrates the source rule that subpackages are packages within packages and can provide multi-level organization.

Package Parts on Disk

The basic physical arrangement has three roles. The package folder provides the container, __init__.py marks that folder as a package, and Python module files hold the modules inside it. When a package contains subpackages, each folder in that package hierarchy must contain its own __init__.py to be recognized as a package.

__init__.py as the Marker

The special filename __init__.py has a structural role: it marks the folder containing it as a package. This is different from an ordinary Python module file. A module is a Python module within the organization; a package is a folder that contains modules and the special marker file.

Module or Package

The distinction is about level and containment. A module is one part of the organization, while a package is a containing folder that groups modules. A package may also contain subpackages, so it can represent a larger level of organization than a single module.

Building Subpackages

To organize a larger project, place related modules in a subpackage and place that subpackage inside an outer package. The outer package and every subpackage must contain __init__.py. This preserves the package hierarchy at every folder level.

1. Choose an outer package folder, such as app.
2. Place __init__.py inside the outer folder.
3. Create subpackage folders for related groups, such as data and reports.
4. Place __init__.py inside every subpackage folder.
5. Place related Python modules inside the appropriate subpackage.

In the generated arrangement, records.py belongs to the data subpackage, while summary.py belongs to the reports subpackage. Both subpackages belong to app. The repeated __init__.py files are what mark app, data, and reports as package folders throughout the hierarchy.

Mistakes in Package Layouts

• Treating a package as though it were only one module file.

  A package is a folder containing Python modules and __init__.py.

  Fix: Look for the containing folder, its __init__.py file, and the modules inside it.

• Adding __init__.py only to the outer folder.

  Every folder in the package hierarchy must contain __init__.py to be recognized as a package.

  Fix: Place __init__.py in the outer package and in each subpackage folder.

• Putting related modules at unrelated levels.

  Packages and subpackages are intended to organize modules into a hierarchy.

  Fix: Group related modules within an appropriate package or subpackage.

Practice the Structure

Structure Checklist

1. A package is a folder containing Python modules and a special __init__.py file.
2. The __init__.py file marks its containing folder as a package.
3. Packages organize modules into a hierarchy, and subpackages are packages within packages.
4. Every folder in a package hierarchy must contain __init__.py to be recognized as a package.
5. A useful organization pattern is variables in functions, functions in modules, and modules in packages.