Modules and packages

So far each program has lived in one file. Real projects split code across many files so that each part has a clear job, can be tested on its own and can be reused. In Python, every .py file is a module, and a folder of modules is a package. The import statement connects them.

This unit explains what import really does: where Python looks for a module, why a module's code runs only once, and how a file with an unlucky name can break a library. It ends with a project layout that many Python projects use.

You will be able to:

• Import modules in each common form and explain what a module object and its namespace are
• Guard script only code with if __name__ == "__main__":
• Explain how Python finds a module through sys.path, and diagnose a file that shadows a library
• Build a package, use absolute and relative imports, and run a module with python -m
• Use the module cache, importlib.reload and __all__ correctly, and untangle a circular import
• Describe the src/ project layout and why it helps

Note. The examples in this unit write their module files into the current folder with pathlib and then call sys.path.insert(0, "."). When you run python main.py, Python already puts the script's folder first on sys.path; the extra line makes the examples work however they are started, including with python -I, which leaves that folder out. Give this unit a new, empty folder of its own, so that the files the examples create do not mix with your other practice files.

A module is a file

A module is a file of Python code. Its name is the file name without .py. Importing it runs the file's code once and gives you a module object whose attributes are the names the file defined. Each module has its own global namespace, the mapping from names to objects, so geometry.area and shapes.area can never clash.

math appears in the list because geometry imported it: an import binds a name in the importing module's namespace. The first import also stores the compiled bytecode in a __pycache__ folder next to the file, so later runs start faster (see Introduction to Python).

Import forms

PEP 8 asks for imports at the top of the file, one module per import line, grouped as standard library, then third party, then your own modules. import module keeps the origin visible at every use (math.sqrt); from module import name is shorter when you use a name often.

Scripts and modules: if __name__ == "__main__":

Every module has a __name__ attribute. When a file is imported, __name__ is the module's name. When the same file is run as the program, Python sets __name__ to "__main__". Testing for that lets one file act both as an importable module and as a script. The example also runs the file as a script with subprocess.run, which starts another program and collects its output (A tour of the standard library covers it properly):

Imported, the file defines greet and skips the guarded block. Run with python greetings.py (here through subprocess), it executes the block too. Put a script's work in a main() function and call it from the guard. Tests can then import the module without starting the program, and multiprocessing needs the guard whenever its workers reimport your main module, which they do with the spawn and forkserver start methods. Since Python 3.14 those are the defaults everywhere: spawn on Windows and macOS, forkserver on Linux (see Concurrency).

How Python finds a module

When you write import name, Python first checks the module cache, the dict sys.modules. If the module isn't there, it asks a series of finders: first for built in and frozen modules compiled into the interpreter (such as sys), then for a file or folder called name in each directory on sys.path, in order. The first match wins.

sys.path is a list of folders, normally built from:

1. the folder of the script you ran (for python -m or the REPL, the current folder);
2. the folders in the PYTHONPATH environment variable, if it is set;
3. the standard library folders;
4. site-packages, where pip installs third party packages (inside your virtual environment if one is active).

Your output will differ: it lists your own installation's folders, then the path of the standard library's json package. The -P option and the PYTHONSAFEPATH variable (both since 3.11) stop Python from putting the script's folder or the current folder first.

The shadowing pitfall

Because the script's folder comes first on sys.path, a file in your project with the same name as a library hides that library. A practice file called random.py, json.py, email.py or pandas.py next to your script is the classic case: the next import random anywhere in that program loads your file instead.

The block runs python dice.py in a child process and shortens the full path in the message. Its last two lines delete both files again. If you try this by hand, delete your random.py afterwards: while it sits in a folder, every program there that imports random gets your file instead. Since Python 3.13 the error message suggests the rename itself. A file that imports a third party library with its own name, such as a pandas.py containing import pandas as pd, imports itself, and the message ends with (consider renaming '...pandas.py' if it has the same name as a library you intended to import).

Pitfall. When a well known library suddenly "has no attribute" that the documentation promises, print module.__file__. If it points into your project, rename your file. Never name your own modules after the standard library (sys.stdlib_module_names lists them) or after packages you install.

Packages

A package is a folder of modules that you import with dotted names, such as shop.tax. A regular package contains a file called __init__.py. That file runs when the package is first imported; it may be empty, or it may import names from the submodules to give the package a tidy interface.

Absolute and relative imports

Inside a package, a module can import its neighbours in two ways:

• An absolute import names the full path from the top level package: from shop.tax import add_vat. PEP 8 recommends this form.
• A relative import starts with dots: from .tax import add_vat means "from the tax module in my own package", and from ..utils import helper goes one package up. Relative imports work only inside packages.

Importing shop.pricing first imported the package shop (running __init__.py), then shop.pricing, which pulled in shop.tax. The second import of shop printed nothing: it came from the cache.

Running a module with python -m

A module that uses relative imports can't be run as a file path, because Python then treats it as a lone script outside any package. Run it with python -m and its dotted name from the project folder instead: Python imports the package first, sets __name__ to "__main__", and puts the current folder on sys.path.

You already use this form: python -m pip and python -m venv run modules from installed packages. If a package contains a __main__.py file, python -m package runs it.

Regular and namespace packages

A folder without __init__.py can still be imported. It becomes a namespace package (PEP 420), which may be spread over several folders on sys.path; Python merges them into one package. This lets separately installed distributions add modules to the same package name.

For your own code, use regular packages. A folder that lacks __init__.py by mistake still imports, as a namespace package, but some tools then treat it differently; for example, setuptools' find_packages() skips it.

Modules are cached

Python runs a module's code once per process. Later imports, from any module, find it in sys.modules and return the same object. That makes a module a natural singleton: module level state is shared by everyone who imports it.

loading config appears once, although two modules imported config, and a change made through one importer is visible to the other.

Pitfall. The file you run is the module __main__. If another module imports that file by its own name, the cache has no entry under that name, so the file runs a second time as a separate module with its own copies of every global. Keep the code you run as a script separate from the modules it imports.

importlib.reload

importlib.reload(module) runs the module's current source again inside the same module object. It is handy in the REPL or a notebook after you edit a file, but it has limits: names copied earlier with from module import name keep their old values, and objects created from old classes keep the old classes.

In a program, restart the process rather than reloading modules.

__all__ and star imports

from module import * binds every public name of the module, meaning every name that doesn't start with an underscore. If the module defines a list called __all__, only the names in that list are imported. __all__ doesn't hide anything else: module.name still reaches every attribute.

Avoid from module import * in your code. Readers can't tell where a name came from, linters can't spot undefined names, and imported names silently replace your own or built in ones:

os exports a low level os.open, which replaced the built in open.

Circular imports

A circular import happens when module A imports module B while B, directly or indirectly, imports A. When B asks for a name from A, A has only partly run, so the name may not exist yet:

The traceback ends with ImportError: cannot import name 'DEFAULT_STATUS' from partially initialized module 'orders' (most likely due to a circular import), followed by the file's path. Python had registered orders in sys.modules and started running it; its first line imported customers, which asked orders for a name not yet defined.

Ways out, best first:

1. Move the shared code into a third module that both import. This is almost always the right fix, because a cycle usually means two modules share a concept that deserves its own home.
2. Import the module instead of the name (import orders) and look up orders.DEFAULT_STATUS inside the function, when it is called rather than at import time.
3. Move the import into the function that needs it. This works, but hides the dependency.

Laying out a project: the src/ layout

Once a project becomes an installable package, many projects use the src/ layout: the package lives in a src folder, with tests and configuration beside it.

Because src/ isn't on sys.path by default, you can't import inventory by accident from the project folder. You install the project into your virtual environment instead, usually in editable mode so that code changes take effect without reinstalling, and your tests then run against the package as users will get it:

pyproject.toml describes the project and how to build it; Packaging and tooling covers it, and Testing with pytest covers the tests/ folder. The simpler "flat" layout, with inventory/ directly in the project folder, also works, but it lets tests silently import the local copy instead of the installed one.

Check yourself

1. What is a module, and what is its __name__ when it is imported? When it is run as a script?
2. In what order does Python look for a module when you write import name?
3. You create json.py next to your script, and import requests now fails inside the library. Why?
4. What is the difference between a regular package and a namespace package?
5. Why does python shop/pricing.py fail on from .tax import add_vat while python -m shop.pricing works?
6. Two modules both import config. How many times does config.py run, and what follows from that?
7. What does __all__ control, and what doesn't it control?
8. What is the usual best fix for a circular import?

Suggested answers

1. A file of Python code, loaded as a module object. Imported, __name__ is the module's name (for example "geometry" or "shop.tax"); run as the program, it is "__main__".
2. The module cache sys.modules first; then built in and frozen modules; then each folder on sys.path in order: the script's folder (or the current folder), PYTHONPATH, the standard library and site-packages.
3. Your folder comes first on sys.path, so any import json, including the one inside requests, loads your file instead of the standard library module. Rename the file.
4. A regular package has an __init__.py, lives in one folder and can run package level code. A namespace package has no __init__.py and may be merged from several folders; its __file__ is None.
5. Run as a file path, the module is a lone script with no parent package, so a relative import has nothing to be relative to. python -m imports it as shop.pricing, as part of the package.
6. Once per process. Both importers share one module object, so module level state, such as a settings dict, is shared too.
7. It lists the names that from module import * binds. It doesn't hide anything from module.name access or from explicit from module import name.
8. Move the shared code into a third module that both modules import.

Exercises

These exercises are optional practice. Pick the ones that help you.

1. Script or module. From a block, write textstats.py containing word_count(text), which returns the number of whitespace separated words, and a guarded main block that prints 5 words for the sentence "the quick brown fox jumps". Importing the module must print nothing. Check both: import it and test word_count, then run it with subprocess and compare its output.
2. A tidy package. Create a package units with a module temperature.py that defines to_celsius(f), to_fahrenheit(c) and a private helper _round1(x) that rounds to one decimal place. Make units/__init__.py reexport the two public functions and set __all__. Then from units import * must bind exactly to_celsius and to_fahrenheit, and to_celsius(212) must give 100.0.
3. Find the shadows. Write shadowing_files(folder) that returns the sorted names of .py files directly inside folder whose name, without .py, is a standard library module name (use sys.stdlib_module_names, available since 3.10). For a folder containing random.py, json.py, app.py and notes.txt, it returns ['json.py', 'random.py'].
4. Plugin registry. Write registry.py with a dict PLUGINS and a decorator register(name) that stores the decorated function under name. Write csv_plugin.py and upper_plugin.py, each registering one text processing function when imported. Importing a plugin module twice must not run its registration twice. Check that the registry ends up with exactly two entries, csv and upper, and that calling PLUGINS["upper"]("hi") gives 'HI'.

Solutions

1. Script or module.

The block's only output is the final line, which shows that importing textstats printed nothing.

2. A tidy package.

The star import runs inside a small, otherwise empty module, star_check, so dir() there lists exactly the names it created.

3. Find the shadows.

4. Plugin registry.

If the second import csv_plugin ran the module again, register("csv") would raise ValueError. It doesn't, because the module is taken from sys.modules, and all three modules share the one registry module and its PLUGINS dict.