Concurrency

Many programs spend most of their time waiting: for a web server to answer, a disk to deliver a file, a database to return rows. Others spend their time computing. Concurrency lets a program make progress on several tasks at once, so the waiting overlaps instead of adding up, and parallelism lets it use more than one CPU core for heavy computation.

Python offers three main tools: threads, processes and asyncio. Each fits a different kind of work, and picking the wrong one either gains nothing or adds bugs. This unit shows how each works, where the global interpreter lock (GIL) fits in, and how to choose.

You will be able to:

• Tell concurrency from parallelism, and I/O bound work from CPU bound work
• Explain what the GIL does, and check whether your Python is a free threaded build
• Run threads, spot a race condition and fix it with a Lock
• Use ThreadPoolExecutor and ProcessPoolExecutor, including the __main__ guard that processes need
• Write asyncio code with async/await, gather, TaskGroup, timeout and Semaphore
• Choose the right tool for a workload and measure the difference

Concurrency, parallelism and two kinds of work

Concurrency means several tasks are in progress during the same period: one task waits, another runs. A single cook who puts pasta on to boil and chops vegetables while it cooks is working concurrently. Parallelism means tasks literally run at the same instant on different CPU cores: two cooks chopping at once. Parallelism is one way to achieve concurrency, but not the only one.

What helps depends on what your program spends its time doing:

Measure before you choose. If a program is slow because of one big calculation, running it on many threads will not help; if it is slow because it makes 200 network calls one after another, a faster CPU will not help either.

The GIL and free threaded Python

In the standard CPython build, a lock called the global interpreter lock (GIL) allows only one thread at a time to run Python bytecode. It keeps the interpreter's own data structures, such as reference counts, consistent. Two consequences follow:

• Threads do speed up I/O bound programs. A thread releases the GIL while it waits on a socket, a file or time.sleep(), so other threads run in the meantime. Many C extensions, such as NumPy and hashlib, also release it during long operations.
• Threads do not speed up CPU bound pure Python code. Only one thread computes at a time, so four threads take about as long as one.

The GIL does not make your own code thread safe. It protects the interpreter, not your invariants: a statement such as balance -= amount is several steps (read, subtract, store), and another thread can run between them. You will see this shortly.

Python 3.13 introduced an optional, experimental free threaded build without the GIL (PEP 703). Since 3.14 it is officially supported (PEP 779), but it remains a separate build that you choose to install, for example through an option in the official Windows and macOS installers. On it, threads run Python code in parallel on several cores. Single threaded code runs a little slower, and importing an extension module that is not marked as safe for free threading turns the GIL back on, with a warning. You can check what you are running:

On a standard build your output looks like this. On a free threaded build both answers flip to True and False, unless something has turned the GIL back on. sys._is_gil_enabled() exists since 3.13. Its leading underscore marks it as an implementation detail, but it is the documented way to ask this question.

Note. Python 3.12 gave each subinterpreter its own GIL, and 3.14 added the concurrent.interpreters module and concurrent.futures.InterpreterPoolExecutor to use them. They are a newer, more specialised option that this unit does not cover.

Threads with threading

A thread is a separate flow of execution inside the same process. All threads share the process's memory, so they can read and write the same objects. You create a threading.Thread with a target function and its args, call start() to launch it, and call join() to wait for it to finish.

The function below stands in for a download: time.sleep() waits the way a network call would.

Three waits of 0.2 s took about 0.2 s in total, not 0.6 s, because they overlapped. Notice:

• A thread's target cannot return a value to you, so each thread writes into a shared dictionary. The executors in the next section handle results more neatly.
• Threads finish in an unpredictable order, so the results are sorted before printing.
• Without the join() calls, the main thread would print before the downloads had finished.

A thread created with daemon=True is stopped abruptly when the main program exits. Use that only for background helpers whose work may be lost, never for writing files or data.

Race conditions and locks

A race condition is a bug whose outcome depends on the timing of threads. The classic case is a read, modify, write: two threads read the same value, each computes a new one, and the second write overwrites the first.

Real races are rare and hard to reproduce, which is what makes them dangerous. The example below forces the unlucky timing with a threading.Barrier, which makes both threads wait until both have read the balance:

The final balance is 70 or 50, depending on which thread writes last, but never the correct 20. A threading.Lock fixes it. Only one thread can hold a lock at a time; another thread that tries to acquire it waits until it is released. Using the lock in a with statement releases it even if the block raises an exception. Here 8 threads each make 10,000 deposits:

Keep the locked block small, and never call something slow (such as a network request) while holding a lock. For handing work from one thread to another, queue.Queue is already thread safe and saves you writing locks yourself.

Pitfall. A program with a race condition can pass its tests thousands of times and then fail in production. Code that mutates shared data from several threads needs a lock by design; you cannot test your way to confidence. On free threaded builds races become more likely, so code that was "fine" under the GIL may break.

concurrent.futures: pools of workers

Starting a thread per task and collecting results by hand gets tedious. The concurrent.futures module offers executors: pools of worker threads or processes that you hand tasks to. Both kinds share one interface:

A future is a placeholder for a result that may not exist yet. Using the executor in a with statement waits for all tasks and shuts the pool down at the end.

With submit and as_completed, one failing task does not stop the others: its exception waits inside its future until you call result(). With map, the first failing item raises when you reach its result in the loop.

Processes for CPU bound work

ProcessPoolExecutor runs tasks in separate Python processes. Each process has its own interpreter and its own GIL, so CPU bound work really runs in parallel. The price is overhead: starting processes takes time, and every argument and result is pickled (serialised) to pass between processes. That means the worker function must be defined at the top level of a module, not as a lambda or a nested function.

The function below counts prime numbers by trial division, a deliberately CPU heavy job, and the pool splits the range into four chunks:

pool.map accepts several iterables, like the built in map, so each call gets one start and one stop.

The if __name__ == "__main__": guard is required here, not merely good style. Each worker process starts a fresh interpreter and imports your main module so that it can find count_primes. Without the guard, every worker would run the pool creating code again while importing. Python stops each worker with a RuntimeError saying that a new process was started before the current one finished its bootstrapping phase, and the main program then fails with BrokenProcessPool.

How a worker process starts is called the start method. It explains why the guard matters on every platform:

Your output depends on your operating system. Older tutorials written for Linux often skip the guard, because the old fork default copied the running process instead of reimporting it. That code breaks on Windows, on macOS and on Linux since 3.14.

Pitfall. With spawn, worker functions must be importable from a module file. Functions defined directly in an interactive session or a Jupyter notebook may fail in the workers; move them into a .py module and import them.

asyncio: many waits on one thread

asyncio runs many tasks concurrently on a single thread. Instead of the operating system switching between threads at any moment, each task gives up control explicitly when it has to wait. That point is marked with await, so switches happen only where you can see them.

The vocabulary:

• async def defines a coroutine function. Calling it does not run its body; it returns a coroutine object.
• await something runs something and, if it has to wait, pauses this coroutine and lets the event loop run others.
• The event loop is the scheduler that resumes each coroutine when what it is waiting for is ready.
• asyncio.run(main()) creates an event loop, runs main() to completion and closes the loop. A program normally calls it once.

asyncio.gather runs the coroutines concurrently and returns their results in argument order, whichever finishes first. The total time is that of the slowest one.

Pitfall. A coroutine only gives way at an await. If it calls a blocking function, such as time.sleep() or a non async HTTP library, the whole event loop stops until that call returns, and every other task waits too. Use async libraries, or hand the blocking call to a thread with await asyncio.to_thread(func, *args). Forgetting await is the other classic slip: fetch("a", 1) on its own only creates a coroutine object, and Python warns that it was never awaited.

Task groups and timeouts

asyncio.TaskGroup (since 3.11) is the structured way to run tasks together. Tasks created with tg.create_task() all finish before the async with block ends. If one fails, the group cancels the others and raises an ExceptionGroup, which you handle with except* (see Errors and exceptions). asyncio.timeout() (also 3.11) cancels the code inside it if it runs too long and raises TimeoutError.

The five second task never finished: the group cancelled it as soon as its sibling failed. With plain gather, the other tasks would keep running after a failure unless you cancelled them yourself.

Limiting concurrency with a semaphore

Starting a thousand requests at once can overload a server, get you rate limited, or run out of sockets. An asyncio.Semaphore(n) is a counter that lets at most n tasks into a block at the same time; the rest wait at async with until a place frees up.

Updating in_flight and peak from several tasks is safe here without a lock, because all tasks run on one thread and none of them can be interrupted between two await points.

Real network code works the same way, with an async HTTP client such as the third party httpx or aiohttp packages (see HTTP and web APIs). This sketch fetches the latest version of three packages from PyPI, at most two at a time, with a timeout on every request:

One client is shared by all requests so connections can be reused, raise_for_status() turns an error response into an exception, and the semaphore keeps the load on the server polite.

Context variables

A global variable is shared by every task, so it cannot hold per request information such as a request ID. The contextvars module solves this. Each asyncio task runs in its own copy of the current context, so a ContextVar set inside one task is invisible to the others and to the code that started them.

log() never receives the ID as an argument, yet each line shows the right one, even though the two tasks interleave. Web frameworks and logging libraries use this to attach request details to log lines.

Choosing a tool

Some rules of thumb:

• Make the code correct and simple first; add concurrency only where measurement shows waiting or a busy CPU.
• Prefer executors and TaskGroup to raw threads and loose tasks: they collect results and errors for you.
• Avoid sharing mutable state. Pass data in, return results out, and use a Lock or a queue.Queue only where sharing is unavoidable.
• For CPU bound work on a free threaded build, threads become an option too, as long as your libraries support it.

One workload, three ways

The script below runs the same five jobs, each waiting 0.2 s, one after another, with a thread pool and with asyncio. The timings depend on your machine, so your output will differ slightly:

All three produce the same results in the same order. The sequential version pays for every wait in turn; the other two overlap them. With CPU bound jobs instead of sleeps, the threaded and asyncio versions would be no faster than the sequential one on a standard build, and a ProcessPoolExecutor would be the tool that helps.

Check yourself

1. What is the difference between concurrency and parallelism?
2. On a standard CPython build, why do threads speed up downloads but not a pure Python number crunching loop?
3. The GIL lets only one thread run Python bytecode at a time. Why can balance -= amount still lose updates when run from several threads?
4. Why must code that creates a ProcessPoolExecutor sit under if __name__ == "__main__":?
5. Why can't you pass a lambda to ProcessPoolExecutor.map?
6. What happens to the other tasks if a coroutine calls time.sleep(2) instead of await asyncio.sleep(2)?
7. One task in an asyncio.TaskGroup raises an exception. What happens to the rest?
8. Two asyncio tasks each set the same ContextVar. Why do they not see each other's value?

Suggested answers

1. Concurrency means several tasks make progress over the same period, possibly by taking turns on one core. Parallelism means tasks execute at the same instant on different cores.
2. A thread releases the GIL while it waits for I/O, so the waits overlap. Pure Python computation needs the GIL the whole time, so only one thread computes at once.
3. The statement is several steps: read the value, subtract, store. A thread can be paused between the read and the store, and another thread's update is then overwritten. Protect the whole read, modify, write with a Lock.
4. With the spawn and forkserver start methods, each worker imports the main module. Without the guard, that import would run the pool creating code again in every worker, which Python stops with a RuntimeError.
5. Arguments and functions are pickled to send them to the worker processes, and pickle refers to functions by their importable name. A lambda (or a nested function) has no such name.
6. They all stop. time.sleep() blocks the only thread the event loop runs on, so no other task can run until it returns.
7. The group cancels the remaining tasks, waits for them to finish, and then raises an ExceptionGroup containing the error.
8. Each task runs in its own copy of the context, made when the task is created. Setting the variable changes only that task's copy.

Exercises

These exercises are optional practice. Pick the ones that help you.

1. A thread safe counter. Write a class Counter with an increment() method and a value property, safe to use from many threads. Start 8 threads that each call increment() 10,000 times, and check that the final value is 80,000.

2. Results in input order. Write fetch_all(names, fetch, max_workers), which calls fetch(name) for every name in a thread pool and returns a list of (name, outcome) pairs in the input order. The outcome is the return value, or "error: <message>" if fetch raised an OSError. Test it with a fake fetch that sleeps 0.05 s and raises ConnectionError("refused") for the name "bad":

3. A polite crawler. Write an async function crawl(pages, limit, timeout). pages maps a page name to the seconds it takes to load; simulate loading with asyncio.sleep. At most limit pages may load at once, and a page that takes longer than timeout seconds gives None. Return a dict of results in the order of pages. With {"home": 0.05, "slow": 1.0, "about": 0.05, "blog": 0.05}, limit=2 and timeout=0.2, the result is {'home': 'home loaded', 'slow': None, 'about': 'about loaded', 'blog': 'blog loaded'} and the whole crawl takes well under a second.

4. Sum of squares on several cores. Write sum_of_squares(start, stop), returning the sum of n * n for n in range(start, stop). Use a ProcessPoolExecutor to compute the total for range(0, 2_000_000) in four chunks, and check it against the formula n * (n - 1) * (2 * n - 1) // 6 with n = 2_000_000. Remember the guard.

Solutions

1. A thread safe counter.

The lock covers the whole += 1, which reads, adds and stores. Reading through the lock as well guarantees that value never sees a half finished update, which matters on free threaded builds.

2. Results in input order.

pool.map keeps the input order, so zipping its results with names pairs them correctly. Catching the exception inside the worker means one failure cannot stop the loop over results. ConnectionError is a subclass of OSError, so it is caught. A thread pool is fine for a nested safe_fetch, because nothing is pickled.

3. A polite crawler.

The timeout sits inside the semaphore, so time spent waiting for a free place does not count against a page. The slow page holds one of the two places for 0.2 s while the other three pages share the second place.

4. Sum of squares on several cores.

Python integers never overflow, so the exact total matches the formula. min(..., n) keeps the last chunk inside the range even when n does not divide evenly.