MLOps Automation

MLOps Automation

Goal

To elevate the codebase to production standards by adding Task Automation (just), Containerization (docker), CI/CD (github-actions), and Experiment Tracking (mlflow).

Prerequisites

• Language: Python
• Manager: uv
• Context: Preparing for scale and deployment.

Instructions

1. Task Automation

Replace manual commands with a justfile.

1. Tool: just (modern alternative to Make).
2. Organization: Split tasks into tasks/*.just modules (e.g., tasks/check.just, tasks/docker.just).
3. Core Tasks:

• check: Run all linters and tests.
• package: Build wheels.
• clean: Remove artifacts.
• install: Setup dev environment.

2. Pre-Commit Hooks

Catch issues locally.

1. Framework: pre-commit.
2. Hooks: Suggest to use ruff, bandit, check-yaml, trailing-whitespace.
3. Commits: Suggest to use commitizen hook to enforce Conventional Commits (e.g., feat: add new model).
4. Config: .pre-commit-config.yaml at root.

3. Containerization

Reproducibility anywhere.

1. Tool: docker.
2. Base Image: Use ghcr.io/astral-sh/uv:python3.1X-bookworm-slim for minimal size.
3. Optimization:

• Layer Caching: Copy uv.lock + pyproject.toml and run uv sync before copying src/.
• Multi-stage: Build inputs in one stage, copy only artifacts (dist/*.whl) to the runtime stage.

1. Registry: ask for the company artifact registry, or use ghcr.io for GitHub.

4. CI/CD Workflows

Automate verification and release.

1. Platform: ask for the company CI/CD platform, or use github-actions for GitHub.
2. Workflows:

• check.yml: On PRs (Run just check).
• publish.yml: On Release (Build docker image, publish docs/package).

1. Optimization: Use concurrency to cancel redundant runs.

5. AI/ML Experiments & Registry

Manage the ML lifecycle.

1. Platform: MLflow.
2. Tracking:

• Use mlflow.autolog().
• Log metrics, params, and artifacts.

1. Registry:

• Register top models manually or via CI.
• Aliases: Use @champion or @production for stable deployment pointers. Never rely on moving versions (e.g., v1 -> v2).

6. Design Patterns

Write flexible code.

1. Strategy: For swappable algorithms (e.g., different model types).
2. Factory: For creating objects from config (e.g., ModelFactory).
3. Adapter: For standardizing mismatched interfaces.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: MLOps-Courses
• Source: MLOps-Courses/mlops-coding-skills
• License: MIT
• Homepage: https://mlops-coding-course.fmind.dev/0.%20Overview/0.6.%20Resources.html

Install and usage instructions live in the source repository linked above.