Python Development

Python Development

Guiding Principle

Apply features only when they add clarity, correctness, performance, or security. Prefer simple, intentional solutions (DRY, KISS, YAGNI, Fail Fast).

The Zen of Python (import this)

The canonical aphorisms by Tim Peters (PEP 20). When in doubt about which Python idiom to choose, re-read these. Quote verbatim; do not paraphrase in code reviews.

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

> Run python -c "import this" in any Python REPL to see it.

For per-line operative readings (how to apply each in review), see the Zen section in rules/200-python.mdc.

AI Assistant Guidelines

• Avoid Over-Engineering: Don't recommend boto3 client caching, async/await, or concurrency patterns unless explicitly requested or bottlenecks are evident
• Keep It Simple: Prefer stdlib over complex architectures
• Respect Context: Don't transform a 20-line script into a 200-line framework

Non-negotiables

> [!IMPORTANT] > New Python applications, services, scheduled jobs, CLI tools, and AWS Lambda functions target Python ≥ 3.14. Treat lower runtimes as a reject-in-review issue unless the code is a library / SDK with a stated compatibility commitment.

NN-1: Python ≥ 3.14 for new applications, services, and Lambda functions

Use requires-python = ">=3.14" in pyproject.toml, Python 3.14 in CI, Python 3.14 Docker images, and Python 3.14 Lambda runtimes for new code.

Libraries published to PyPI or shipped to external customers MAY target a lower floor when there is a documented compatibility commitment. The acceptable lower floor is Python 3.11. The PR description must explain the audience, the 3.14 features being deferred, and the planned floor-bump date.

Reject in review:

• New application / service / Lambda with requires-python = ">=3.11" (or lower) and no library-audience justification
• Missing requires-python in pyproject.toml
• New code targeting Python 3.10 or below for any reason
• CI, Docker, or Lambda runtime config pinned below Python 3.14 for new app / service / Lambda code

NN-2: Leading underscores mean non-public API

Do not prefix functions, methods, classes, variables, modules, or packages with _ unless they are intentionally internal implementation details. Public behavior gets public names.

# Public API, public name
def validate_order(order: Order) -> None:
    ...

# Internal helper, module-private name
def _normalize_order_id(raw_order_id: str) -> str:
    ...

Reject in review:

• _process_data(), _validate_input(), or _build_payload() called directly from other modules
• Public classes / modules named _Client, _Service, _helpers, or similar
• Double-underscore methods (__method) unless name-mangling is intentionally required
• Invented dunder names such as __process__ or __validate__
• Leading underscore added merely because the function is small or "helper-ish"

Standards Quick Reference

| Aspect | Standard | |--------|----------| | Python Version | ≥ 3.14 | | Shebang | #!/usr/bin/env -S uv run | | Formatting | 4-space indents, 120 char line length | | Linting | black, ruff, pylint (≥9.0) | | Type Hints | Strict typing required | | Docstrings | Google-style | | Package Manager | uv (preferred) |

Documentation Contract

For new scripts, place the module-level docstring immediately below the shebang, with one blank line between them. The module docstring must explain the script's purpose, provide a short overview, include a numbered workflow when there are multiple steps, and include CLI usage examples when the file is meant to be run directly. AWS Lambda handlers and import-only modules do not need CLI usage examples.

#!/usr/bin/env -S uv run
"""
Process user export files and publish normalized records.

This script reads a JSON export, validates each record, writes a cleaned
CSV file, and optionally uploads the result to object storage.

Workflow:
1. Parse command-line arguments.
2. Validate input file and output directory.
3. Load and validate JSON records.
4. Write normalized CSV output.
5. Upload the result when --upload is set.

Usage:
    python process_users.py --input users.json --output users.csv
    python process_users.py --input users.json --output users.csv --upload
"""

Every public function, class, and method must use Google-style docstrings. Put the description on a new line after the opening triple quotes, then document arguments, return value, raised exceptions, and examples when helpful.

def load_users(input_path: Path) -> list[User]:
    """
    Load user records from a JSON file.

    Args:
        input_path (Path): Path to the JSON file containing user records.

    Returns:
        list[User]: Validated user records parsed from the input file.

    Raises:
        FileNotFoundError: If the input file does not exist.
        ValueError: If the file contains invalid JSON or invalid user records.

    Example:
        users = load_users(Path("users.json"))
    """

Inline comments are for complex logic, non-obvious tradeoffs, or external constraints. Do not comment obvious assignments or restate function names.

Type Hints

Use built-in types instead of typing module:

# Modern Python 3.14+
def process(items: list[str], config: dict[str, int] | None = None) -> tuple[str, int]:
    ...

# Avoid (old style)
from typing import List, Dict, Optional, Tuple
def process(items: List[str], config: Optional[Dict[str, int]] = None) -> Tuple[str, int]:
    ...

Python 3.14 Features (Released Oct 2025)

Template String Literals (PEP 750)

Safe custom string processing with t-strings:

# SQL query with safe parameterization
query = t"SELECT * FROM users WHERE id = {user_id}"

# HTML generation with auto-escaping
html = t"{user_input}"

# Custom formatting
config = t"server={host}:{port}"

Deferred Annotation Evaluation (PEP 649)

Annotations are no longer evaluated eagerly, improving startup performance:

# Annotations stored in __annotate__ function
# Evaluated only when inspect.get_annotations() is called
def process(data: ComplexType) -> Result:
    """Annotations evaluated lazily, not at import time."""
    ...

Free-Threading Support (PEP 779)

True parallelism without the GIL (experimental):

# Enable with: python3.14t (free-threaded build)
# Performance penalty reduced to 5-10% for single-threaded code
import concurrent.futures

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(cpu_intensive_task, items)

datetime Improvements

Direct parsing of date and time strings:

from datetime import date, time

# Python 3.14+
d = date.fromisoformat("2025-10-07")
t = time.fromisoformat("14:30:00")

UUID7 and UUID8 Support

Modern UUID versions with better properties:

import uuid

# UUID7: Time-ordered, sortable (recommended for databases)
id = uuid.uuid7()

# UUID8: Custom format
id = uuid.uuid8(bytes16)

Code Structure

#!/usr/bin/env -S uv run
"""
Module purpose and overview.

Workflow:
1. Load configuration
2. Validate inputs
3. Process data
"""

# Standard library
import logging

# Third-party
import requests

# Local
from utils import helper

def helper_function() -> None:
    """Helper functions first."""
    pass

def main() -> int:
    """Main function last."""
    return 0

if __name__ == "__main__":
    sys.exit(main())

Key Patterns

Defensive Programming

# Validate inputs early (fail-fast)
def process_user(user_id: str | None) -> User:
    if user_id is None:
        raise ValueError("user_id is required")
    # Continue processing...

No Mutable Defaults

# BAD
def foo(items: list[str] = []):
    ...

# GOOD
def foo(items: list[str] | None = None):
    items = items or []
    ...

Error Handling

# Re-raise with context at boundaries
try:
    data = load_database()
except Exception as e:
    raise RuntimeError(f"Failed to load database: {e}") from e

Logging Setup

import logging
import time

def setup_logger(name: str) -> logging.Logger:
    logger = logging.getLogger(name)
    logger.setLevel(logging.INFO)
    
    class UTCFormatter(logging.Formatter):
        converter = time.gmtime
    
    handler = logging.StreamHandler()
    handler.setFormatter(UTCFormatter("%(asctime)s - %(levelname)s - %(message)s"))
    logger.addHandler(handler)
    return logger

logger = setup_logger(__name__)

Package Management (uv)

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Project setup
uv init my-project
uv add boto3 pydantic
uv add --dev pytest black ruff
uv sync

# Run script
uv run python main.py

Pydantic Validation

from pydantic import BaseModel, field_validator

class User(BaseModel):
    name: str
    age: int
    
    @field_validator("age")
    @classmethod
    def valid_age(cls, v: int) -> int:
        if v  150:
            raise ValueError("Invalid age")
        return v

Quick Checklist

• [ ] Shebang line with uv
• [ ] Type hints on all functions
• [ ] Google-style docstrings
• [ ] Imports grouped and sorted
• [ ] if __name__ == "__main__": guard
• [ ] Logging configured
• [ ] Input validation
• [ ] Specific exception handling
• [ ] black and ruff pass
• [ ] pylint score ≥ 9.0

Detailed References

• Code Quality Tools: See [references/code-quality-tools.md](references/code-quality-tools.md) for black, ruff, isort, mypy, pylint configuration
• Package Management: See [references/package-management.md](references/package-management.md) for uv, isort, dependency management
• Standard Library: See [references/standard-library.md](references/standard-library.md) for pathlib, textwrap, contextlib, dataclasses, enum, typing, secrets, heapq, graphlib
• Logging & Observability: See [references/logging-observability.md](references/logging-observability.md) for logger setup, structured logging, log levels
• Security & Validation: See [references/security-validation.md](references/security-validation.md) for bandit, input validation, Pydantic, regex patterns
• Error Handling: See [references/error-handling.md](references/error-handling.md) for exception handling, retry patterns, custom exceptions
• Performance Optimization: See [references/performance-optimization.md](references/performance-optimization.md) for profiling, optimization examples, concurrency guide
• Troubleshooting & Debugging: See [references/troubleshooting-debugging.md](references/troubleshooting-debugging.md) for pdb, profiling, common issues
• Design Patterns: See [references/design-patterns.md](references/design-patterns.md) for decorator, factory, singleton, strategy patterns
• CLI & User Experience: See [references/cli-user-experience.md](references/cli-user-experience.md) for argparse, typer, rich, UX best practices
• Modern Python Features: See [references/modern-python.md](references/modern-python.md) for match-case, walrus operator, dataclasses, functional patterns
• Async & Concurrency: See [references/async-concurrency.md](references/async-concurrency.md) for asyncio, threading, multiprocessing
• Testing Patterns: See [references/testing-patterns.md](references/testing-patterns.md) for pytest, mocking, fixtures
• AWS Lambda: See [references/aws-lambda.md](references/aws-lambda.md) for boto3 patterns, Lambda best practices
• AWS & Boto3: See [references/aws-boto3.md](references/aws-boto3.md) for client configuration, error handling, pagination, Lambda patterns

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: d-padmanabhan
• Source: d-padmanabhan/agent-engineering-handbook
• License: MIT

Install and usage instructions live in the source repository linked above.