Skip to main content
Advertisement

Practical Tips — Exception Logging, Exception Chaining, Defensive Programming

Exception handling goes beyond simply catching errors. It must be designed so that problems can be quickly diagnosed and recovered from in production. This chapter covers advanced patterns frequently used in real-world code.

Exception Logging — logging Module + exc_info=True​

Using the exc_info=True parameter of the logging module includes the full exception traceback in the log.

import logging

# Logger configuration
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)

def process_data(data: dict) -> dict:
    try:
        result = transform(data)
        return result
    except KeyError as e:
        # exc_info=True: includes full traceback
        logger.error("Missing required key: %s", e, exc_info=True)
        return {}
    except Exception:
        # logger.exception: shortcut that automatically includes exc_info=True
        logger.exception("Unexpected error occurred")
        raise

logger.exception() behaves identically to logger.error(..., exc_info=True). Use it only inside exception handlers.

Structured Logging (structlog pattern)​

import logging

logger = logging.getLogger(__name__)

def process_order(order_id: int, user_id: int) -> None:
    try:
        _execute_order(order_id)
    except ValueError as e:
        logger.error(
            "Order processing failed",
            extra={
                "order_id": order_id,
                "user_id": user_id,
                "error": str(e),
            },
            exc_info=True,
        )
        raise

Exception Chaining — raise X from Y​

The raise X from Y syntax raises a new exception while explicitly linking the cause exception (Y). Both exceptions appear in the traceback.

class DatabaseError(Exception):
    pass

def load_user(user_id: int) -> dict:
    try:
        return db.execute("SELECT * FROM users WHERE id = ?", (user_id,))
    except sqlite3.OperationalError as e:
        # Wrap low-level exception into a domain exception
        raise DatabaseError(f"Failed to retrieve user {user_id}") from e

Traceback output:

sqlite3.OperationalError: no such table: users

The above exception was the direct cause of the following exception:

DatabaseError: Failed to retrieve user 42

Without from e, the message reads "During handling of the above exception, another exception occurred:" (implicit chaining).

Suppressing the Cause — raise X from None​

To hide the original exception and show only the new one, use from None. This is useful when you don't want to expose sensitive internal details (DB queries, connection info, etc.) to the outside.

def get_config(key: str) -> str:
    try:
        return _internal_config[key]
    except KeyError:
        # from None: hides the internal KeyError traceback
        raise KeyError(f"Config key '{key}' not found") from None
KeyError: "Config key 'DATABASE_URL' not found"
# Internal KeyError: 'DATABASE_URL' is not exposed

EAFP vs LBYL Philosophy​

The Python community has two defensive programming philosophies.

EAFP (Easier to Ask Forgiveness than Permission)​

"It's easier to ask forgiveness than permission" — try first, then handle failures.

# EAFP style — Pythonic way
def get_user_age(user: dict) -> int:
    try:
        return int(user["age"])
    except (KeyError, ValueError):
        return 0

LBYL (Look Before You Leap)​

"Look before you leap" — check conditions before executing.

# LBYL style — common in C/Java
def get_user_age(user: dict) -> int:
    if "age" in user and isinstance(user["age"], (int, str)):
        try:
            return int(user["age"])
        except ValueError:
            return 0
    return 0

Python prefers EAFP. In concurrent environments (multi-threaded), EAFP also prevents TOCTOU (Time of Check to Time of Use) race conditions.

# LBYL — race condition possible in multi-threaded code
if os.path.exists(path):      # Check
    os.remove(path)            # Another thread may have already deleted it

# EAFP — safe
try:
    os.remove(path)
except FileNotFoundError:
    pass

Defensive Programming Patterns​

Pattern 1 — Precondition Checks (Guard Clause)​

def process_payment(amount: float, user_id: int) -> None:
    # Check preconditions at the start of the function
    if amount <= 0:
        raise ValueError(f"Payment amount must be positive: {amount}")
    if user_id <= 0:
        raise ValueError(f"Invalid user ID: {user_id}")

    # Core logic after validation passes
    _charge_payment(amount, user_id)

Pattern 2 — Null Object Pattern to Guard Against None​

from typing import Protocol

class UserRepository(Protocol):
    def find(self, user_id: int) -> dict | None: ...

class NullUser:
    """Null Object returned when user cannot be found"""
    name = "Anonymous"
    email = ""
    is_authenticated = False

    def __bool__(self) -> bool:
        return False

def get_user(repo: UserRepository, user_id: int):
    user = repo.find(user_id)
    return user if user is not None else NullUser()

# Safe to use without None checks
user = get_user(repo, 999)
print(user.name)   # "Anonymous" — no AttributeError

Pattern 3 — Result Type (Result Pattern)​

from dataclasses import dataclass
from typing import Generic, TypeVar

T = TypeVar("T")
E = TypeVar("E", bound=Exception)

@dataclass
class Ok(Generic[T]):
    value: T
    ok: bool = True

@dataclass
class Err(Generic[E]):
    error: E
    ok: bool = False

Result = Ok[T] | Err[E]

def parse_int(value: str) -> Result:
    try:
        return Ok(int(value))
    except ValueError as e:
        return Err(e)

# Handle errors as values without exceptions
result = parse_int("abc")
if result.ok:
    print(f"Parse success: {result.value}")
else:
    print(f"Parse failure: {result.error}")

Practical — Production Error Handling Patterns​

Pattern 1 — Global Exception Handler​

import logging
import sys
import traceback

logger = logging.getLogger("global")

def global_exception_handler(exc_type, exc_value, exc_traceback):
    """Catches and logs unhandled exceptions."""
    if issubclass(exc_type, KeyboardInterrupt):
        sys.__excepthook__(exc_type, exc_value, exc_traceback)
        return
    logger.critical(
        "Unhandled exception",
        exc_info=(exc_type, exc_value, exc_traceback)
    )

sys.excepthook = global_exception_handler

Pattern 2 — Decorator-based Exception Handling​

import functools
import logging
from typing import Callable, TypeVar

F = TypeVar("F", bound=Callable)

def handle_errors(
    *exception_types: type[Exception],
    default=None,
    log_level: int = logging.ERROR,
):
    """Decorator that automatically handles function exceptions"""
    def decorator(func: F) -> F:
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            try:
                return func(*args, **kwargs)
            except exception_types as e:
                logging.log(
                    log_level,
                    "%s error: %s",
                    func.__qualname__,
                    e,
                    exc_info=True,
                )
                return default
        return wrapper  # type: ignore
    return decorator

# Usage
@handle_errors(FileNotFoundError, PermissionError, default=[])
def load_items(path: str) -> list:
    with open(path) as f:
        import json
        return json.load(f)

Pattern 3 — Circuit Breaker​

import time
from enum import Enum, auto

class CircuitState(Enum):
    CLOSED = auto()    # Normal
    OPEN = auto()      # Tripped
    HALF_OPEN = auto() # Recovery attempt

class CircuitBreaker:
    """
    Blocks requests when consecutive failures to an external service occur.
    """

    def __init__(
        self,
        failure_threshold: int = 5,
        recovery_timeout: float = 30.0,
    ) -> None:
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.last_failure_time: float | None = None
        self.state = CircuitState.CLOSED

    def call(self, func, *args, **kwargs):
        if self.state == CircuitState.OPEN:
            if self._should_attempt_reset():
                self.state = CircuitState.HALF_OPEN
            else:
                raise RuntimeError("Circuit breaker OPEN — request blocked")

        try:
            result = func(*args, **kwargs)
            self._on_success()
            return result
        except Exception as e:
            self._on_failure()
            raise

    def _on_success(self):
        self.failure_count = 0
        self.state = CircuitState.CLOSED

    def _on_failure(self):
        self.failure_count += 1
        self.last_failure_time = time.time()
        if self.failure_count >= self.failure_threshold:
            self.state = CircuitState.OPEN

    def _should_attempt_reset(self) -> bool:
        return (
            self.last_failure_time is not None
            and time.time() - self.last_failure_time > self.recovery_timeout
        )

Expert Tips​

Tip 1 — Capture exception traceback as a string

import traceback

try:
    1 / 0
except ZeroDivisionError:
    tb_str = traceback.format_exc()
    # Save to log or send as an alert later
    send_alert(tb_str)

Tip 2 — __cause__ and __context__ attributes

try:
    raise ValueError("cause")
except ValueError as e:
    exc = RuntimeError("effect")
    exc.__cause__ = e      # Same effect as raise X from Y
    raise exc

# Access cause from exception object
try:
    ...
except RuntimeError as e:
    print(e.__cause__)      # Explicit chaining
    print(e.__context__)    # Implicit chaining

Tip 3 — Soft warnings with the warnings module

When you need to notify users but raising an exception is too harsh, use warnings.warn().

import warnings

def deprecated_function():
    warnings.warn(
        "deprecated_function will be removed in v2.0. Use new_function() instead.",
        DeprecationWarning,
        stacklevel=2,
    )
    return new_function()

Tip 4 — Pattern for logging and re-raising exceptions

# Log but preserve the exception
try:
    risky()
except Exception:
    logger.exception("risky() failed")
    raise  # Re-raise original exception (preserves stack info)

# Never do this
try:
    risky()
except Exception as e:
    logger.error(str(e))
    raise e  # ← Stack trace is reset to this line!
Advertisement
Advertisement