Decorators Done Right

Welcome to the third course in the Advanced Python Language Features path! By completing the previous two courses on the Python Data Model and Class Machinery, you've built a solid foundation in designing custom types, implementing protocols, and creating extensible systems. Now, we're ready to explore functional programming patterns and modern control flow structures that will make your code more expressive and declarative.

In this course, we'll master four essential patterns: decorators, generic functions with single dispatch, structural pattern matching, and composable error handling. Today's lesson focuses on decorators done right: we'll build a configurable retry mechanism with exponential backoff and jitter, learning how to preserve function metadata, control exception handling, and leverage advanced typing features for type-safe decorator factories. By the end of this lesson, you'll be able to write decorators that feel professional and production-ready.

Decorators are powerful tools for adding reusable behavior to functions without modifying their core logic. However, many decorators in the wild have issues: they lose function metadata, break type checking, or lack configurability. A production-grade decorator must handle these concerns (and potentially more) gracefully.

Consider a retry decorator for network operations. A naive implementation might catch all exceptions and retry indefinitely, which could mask permanent failures or overwhelm a failing service. A robust solution needs:

• Exponential backoff to reduce load on struggling systems
• Jitter to prevent synchronized retry storms
• Selective exception handling to distinguish transient from permanent failures
• Preserved function identity so debugging and introspection work correctly
• Type safety so IDEs and type checkers understand the decorated function

These requirements transform a simple wrapper into a sophisticated, reusable component.

Let's start with the simplest possible retry decorator to establish the core pattern. A retry decorator wraps a function and catches exceptions, attempting the call again after a delay:

import time
from functools import wraps

def simple_retry(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        for attempt in range(3):
            try:
                return func(*args, **kwargs)
            except Exception:
                if attempt == 2:
                    raise
                time.sleep(1.0)
        return None  # unreachable, but satisfies type checker
    return wrapper

This basic decorator tries the function up to three times, waiting one second between attempts. The @wraps(func) line preserves the original function's metadata. However, this implementation is inflexible: the retry count and delay are hardcoded, and it catches all exceptions indiscriminately.

The functools.wraps decorator is critical for maintaining the original function's identity. Without it, decorated functions lose their name, docstring, and other attributes:

from functools import wraps

def retry(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@retry
def unstable():
    """Fetch data from unreliable service."""
    return "data"

print(unstable.__name__)
print(unstable.__doc__)

unstable
Fetch data from unreliable service.

The @wraps(func) call copies attributes like __name__, __doc__, __module__, __qualname__, and __annotations__ from the original function to the wrapper. This ensures that tools relying on introspection, such as debuggers, documentation generators, and testing frameworks, work correctly.

Modern Python decorators should preserve type information for static analysis. The ParamSpec and TypeVar types allow us to capture and forward both parameter types and return types accurately:

from typing import Callable, ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")

def retry(func: Callable[P, T]) -> Callable[P, T]:
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
        return func(*args, **kwargs)
    return wrapper

Here's what each type variable accomplishes:

• P = ParamSpec("P") captures the parameter specification of the decorated function, including positional and keyword arguments
• T = TypeVar("T") captures the return type
• Callable[P, T] represents a function that accepts parameters matching P and returns type T
• P.args and P.kwargs allow the wrapper to accept the same parameters as the original function

This signature tells type checkers that the decorator returns a function with exactly the same signature as the input function, preserving full type safety.

Instead of a simple decorator, we need a decorator factory that accepts configuration parameters. This allows different retry policies for different functions:

def retry(
    *,
    retries: int = 3,
    delay: float = 0.05,
    backoff: float = 2.0,
    jitter: float = 0.2,
    exceptions: tuple[type[BaseException], ...] = (Exception,),
) -> Callable[[Callable[P, T]], Callable[P, T]]:
    def decorator(func: Callable[P, T]) -> Callable[P, T]:
        @wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            # retry logic here
            pass
        return wrapper
    return decorator

The factory pattern creates a three-layer structure:

• retry(...) accepts configuration and returns a decorator
• decorator(func) accepts the function to decorate and returns the wrapper
• wrapper(*args, **kwargs) is the actual decorated function that gets called

The keyword-only parameters (*,) enforce explicit configuration, making the decorator's behavior clear at the call site. Note that, in this decorator, retries means the number of additional attempts after the initial call; that is, retries=3 allows up to 4 total attempts.

Now let's implement the core retry loop with exponential backoff and jitter. The wrapper needs to track attempts and calculate increasing delays:

import random
import time
from typing import cast

def retry(
    *,
    retries: int = 3,
    delay: float = 0.05,
    backoff: float = 2.0,
    jitter: float = 0.2,
    exceptions: tuple[type[BaseException], ...] = (Exception,),
    predicate: Callable[[BaseException], bool] | None = None,
) -> Callable[[Callable[P, T]], Callable[P, T]]:
    def decorator(func: Callable[P, T]) -> Callable[P, T]:
        @wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            attempt = 0
            while True:
                try:
                    return func(*args, **kwargs)
                except exceptions as e:
                    if predicate is not None and not predicate(e):
                        raise
                    if attempt >= retries:
                        raise
                    base = delay * (backoff ** attempt)
                    sleep_for = base + (random.uniform(0, base * jitter) if jitter else 0.0)
                    time.sleep(max(0.0, sleep_for))
                    attempt += 1
        return cast(Callable[P, T], wrapper)
    return decorator

The retry logic works as follows:

• We attempt the function call inside a try block
• If it succeeds, we return immediately
• If it raises one of the specified exceptions, we check the predicate (if provided)
• If the predicate returns False or we've exhausted retries, we re-raise the exception
• Otherwise, we calculate a sleep duration using exponential backoff and jitter, then try again

The cast(Callable[P, T], wrapper) helps the type checker understand that wrapper matches the signature of the decorated function. Note that this decorator implementation is designed for synchronous functions: it won't work correctly with async functions because it uses time.sleep() (which blocks the event loop) and doesn't handle awaitable return values. But don't worry, we'll be encountering asynchronous functions soon enough, in the next course of this path!

The exceptions parameter lets us specify which exception types should trigger retries. This is crucial for distinguishing transient failures from permanent errors:

class TransientError(Exception):
    pass

class PermanentError(Exception):
    pass

@retry(retries=4, delay=0.01, exceptions=(TransientError,))
def permanent_failure() -> str:
    raise PermanentError("never retry this")

Here, only TransientError instances will be caught and retried. If the function raises PermanentError, it propagates immediately without any retry attempts. This prevents wasting time and resources on operations that cannot succeed through retrying.

Sometimes we need finer-grained control: even within a retryable exception type, some instances should not be retried. The predicate parameter accepts a callable that inspects the exception and returns whether to retry:

class TransientError(Exception):
    def __init__(self, msg: str, *, retryable: bool = True) -> None:
        super().__init__(msg)
        self.retryable = retryable

@retry(
    retries=3,
    delay=0.01,
    exceptions=(TransientError,),
    predicate=lambda e: getattr(e, "retryable", False),
)
def sometimes_nonretryable(flips: list[bool]) -> str:
    if not flips:
        return "DONE"
    retryable = flips.pop(0)
    raise TransientError("fatal" if not retryable else "retryable", retryable=retryable)

The predicate checks the retryable attribute on the exception. If it's False, the decorator re-raises immediately, even though the exception type matches. This pattern is useful when error responses contain information about whether retrying makes sense.

Let's see the decorator in action with a function that succeeds after a specific number of attempts:

def make_unstable_svc(success_after: int) -> Callable[[], str]:
    calls = {"n": 0}

    @retry(retries=5, delay=0.01, backoff=2.0, jitter=0.1, exceptions=(TransientError,))
    def unstable() -> str:
        calls["n"] += 1
        if calls["n"] < success_after:
            raise TransientError("temporary failure", retryable=True)
        return f"OK-{calls['n']}"
    unstable.calls = calls  # type: ignore[attr-defined]
    return unstable

svc = make_unstable_svc(success_after=3)
print(svc())
print(svc.__name__)
print(svc.calls["n"])  # type: ignore[index]

OK-3
unstable
3

The service fails twice with TransientError, then succeeds on the third call. Notice that svc.__name__ is "unstable", not "wrapper", demonstrating that @wraps preserved the function's identity. The call count confirms three attempts were needed.

Now let's test permanent failures and predicate-based filtering:

try:
    permanent_failure()
except Exception as e:
    print(type(e).__name__)

flips = [True, False, True]
try:
    sometimes_nonretryable(flips)
except Exception as e:
    print(str(e))

PermanentError
fatal

The permanent_failure function raises PermanentError, which isn't in the exceptions tuple, so it propagates immediately without retries. The sometimes_nonretryable function raises a TransientError with retryable=False on the second call, and the predicate causes it to propagate immediately, showing the message "fatal".

Finally, let's verify that the decorator respects the retry limit:

fail_fast = make_unstable_svc(success_after=999)
try:
    fail_fast()
except Exception as e:
    print(type(e).__name__)

TransientError

Even though the service would eventually succeed, it never will within our 5-retry limit, so the exception is re-raised after the final attempt.

You've now built a production-grade retry decorator that handles the complexities of real-world error recovery: exponential backoff with jitter prevents overwhelming failing services, selective exception handling distinguishes transient from permanent errors, and predicate-based filtering provides fine-grained control. By preserving function metadata and using modern typing features, the decorator integrates seamlessly into typed codebases.

This decorator pattern forms the foundation for many functional programming techniques. You've learned how to build decorator factories, manage state across retries, and compose behavior around function calls. These skills will serve you well as you explore more advanced functional patterns.

Now it's time to put these concepts into practice. Head to the practice section, where you'll configure retry behaviors, implement predicate logic, and extend the decorator with new features!