9. Context Managers

Yes! Our journey is complete, we are where I wanted to be. Context managers are one of my favorites, and a little underused, especially in user code, when they are really easy to both write and use (while decorators, for comparison, are really easy to use but a bit tricky to write). A context manager has a specific purpose.

A context manager is used for what I call “action at a distance”. It lets you schedule an action for later that is sure to always happen (unless you get a segfault, or exit a really nasty way). This is likely the most famous context manager:

with open(...) as f:
    txt = f.readlines()

When you enter the with block, __enter__ is called and the result is assigned to the as target, if there is one. Then when you leave the block, __exit__ is called. If you leave via an exception, __exit__ gets special arguments that let you even decide what to do based on that the exception is - or even handle the exception and continue. contextlib has several simple context managers, like redirect_stdout, redirect_stderr, and suppress:

import contextlib

with contextlib.suppress(ZeroDivisionError):
    1 / 0
    print("This is never reached")

But the real star of contextlib is contextmanager, which is a decorator that makes writing context managers really easy. You use “yield” to break the before and after code. Let’s try one of my favorites, a timer context manager:

import time


@contextlib.contextmanager
def timer():
    old_time = time.monotonic()
    try:
        yield
    finally:
        new_time = time.monotonic()
        print(f"Time taken: {new_time - old_time} seconds")

with timer():
    print("Start")
    time.sleep(1.5)
    print("End")

Start

End
Time taken: 1.501692276 seconds

As an extra bonus, contextmanager uses ContextDecorator, so the objects it makes can also be used as Decorators!

@timer()
def long_function():
    print("Start")
    time.sleep(1.5)
    print("End")


long_function()

Start

End
Time taken: 1.5017058960000043 seconds

Just a quick word on this: if you are coming from a language like JavaScript or Ruby, you might be thinking these look like blocks/Procs/lambdas. They are not; they are unscoped, and you cannot access the code inside the with block from the context manager (unlike a decorator, too). So you cannot create a “run this twice” context manager, for example. They are only for action-at-a-distance.

Pretty much everything in the contextlib module that does not have the word async in it is worth learning. contextlib.closing turns an object with a .close() into a context manager, and contextlib.ExitStack lets you nest context managers without eating up massive amounts of whitespace.

9.1. Quick note: Async

Everything we’ve been doing has built on itself, and we seemed to be going somewhere; the pinnacle of this direction was actually not context managers, but async/await. All of this feeds into async/await, which was formally introduced as a language component in Python 3.6. However, we did skip a necessary step; we didn’t talk about generators (iterators can actually “send” values in, not just produce them, but there’s no specific construct for doing that, like there is for consuming values in a for loop). The main reason we didn’t try to reach async though is that I’ve never found a great use for it in scientific programming; it is much more intrusive than normal threading, it doesn’t really “live” side-by-side with normal synchronous programming all that well (it’s better now, though), and the libraries for it are a little young. Feel free to investigate on your own, though! I’ve also discussed the mechanisms behind it in detail in my blog a few years ago.

Let’s whet your appetite with a quick example, though:

import asyncio


# This is an "async" function, like a generator
async def slow(t: int) -> int:
    print(f"About to sleep for {t} seconds")
    await asyncio.sleep(t)
    print(f"Slept for {t} seconds")
    return t


# Gather runs its arguments in parallel when awaited on
await asyncio.gather(slow(3), slow(1), slow(2))

# Only works if running in an eventloop already, like IPython or with python -m asyncio
# Otherwise, use: asyncio.run(...)

About to sleep for 3 seconds
About to sleep for 1 seconds
About to sleep for 2 seconds

Slept for 1 seconds

Slept for 2 seconds

Slept for 3 seconds

[3, 1, 2]

Notice there are no locks! We don’t have to worry about printing being overleaved, because it’s not running at the same time. Only the explicit “await” lines “wait” at the same time!

Once we start using Python 3.11 (probably early-mid 2023, based on Pyodide), an asyncio section using TaskGroups will likely be added.