Local Ai Models

iOS On-Device AI Models

Production-ready guide for implementing on-device AI models in iOS apps using Apple's Foundation Models framework and MLX Swift.

When to Use This Skill

• Implementing local LLM inference in iOS apps
• Building chat interfaces with Foundation Models
• Integrating Vision Language Models (VLMs)
• Adding text embeddings or image generation
• Implementing tool/function calling with LLMs
• Managing multi-turn conversations
• Optimizing memory usage for on-device models
• Supporting internationalization in AI features

Core Principles

1. Availability First - Always check model availability before initialization
2. Stream Responses - Provide progressive UI updates for better UX
3. Session Persistence - Reuse LanguageModelSession for multi-turn conversations (Foundation Models)
4. Memory Awareness - Use quantized models and monitor memory usage
5. Async Everything - Load models asynchronously, never block the main thread
6. Locale Support - Use supportsLocale(_:) and locale instructions for Foundation Models

Quick Reference

Framework Comparison

| Topic | Guide | | ---------------------------------- | ----------------------------------------------------------- | | Framework comparison and selection | [framework-selection.md](references/framework-selection.md) |

Foundation Models (Apple's Framework)

| Topic | Guide | | ------------------------------- | ----------------------------------------------------------------------------------- | | Setup and configuration | [foundation-models/setup.md](references/foundation-models/setup.md) | | Chat patterns and conversations | [foundation-models/chat-patterns.md](references/foundation-models/chat-patterns.md) |

MLX Swift (Advanced Features)

| Topic | Guide | | ---------------------------------------- | --------------------------------------------------------------------------- | | Setup and configuration | [mlx-swift/setup.md](references/mlx-swift/setup.md) | | Chat patterns with custom models | [mlx-swift/chat-patterns.md](references/mlx-swift/chat-patterns.md) | | Vision Language Models (VLMs) | [mlx-swift/vision-patterns.md](references/mlx-swift/vision-patterns.md) | | Tool calling, embeddings, structured gen | [mlx-swift/advanced-patterns.md](references/mlx-swift/advanced-patterns.md) | | Model quantization with MLX-LM | [mlx-swift/quantization.md](references/mlx-swift/quantization.md) |

Shared (Both Frameworks)

| Topic | Guide | | ------------------------------- | --------------------------------------------------------------- | | Best practices and optimization | [shared/best-practices.md](references/shared/best-practices.md) | | Error handling and recovery | [shared/error-handling.md](references/shared/error-handling.md) | | Testing strategies | [shared/testing.md](references/shared/testing.md) |

Quick Decision Trees

Which framework should I use?

Do you need advanced features like:
- Vision Language Models (VLMs)
- Image generation
- Custom models beyond the system model
├── Yes → MLX Swift (references/mlx-swift/)
└── No → Is this a standard chat interface?
    ├── Yes → Foundation Models (simpler, recommended)
    └── No → Check framework-selection.md for guidance

Where should I start?

New to on-device AI?
└── Start with Foundation Models:
    1. Read framework-selection.md
    2. Follow foundation-models/setup.md
    3. Implement foundation-models/chat-patterns.md

Need advanced features?
└── Use MLX Swift:
    1. Read framework-selection.md
    2. Follow mlx-swift/setup.md
    3. Choose pattern:
       - Chat: mlx-swift/chat-patterns.md
       - Vision: mlx-swift/vision-patterns.md
       - Advanced: mlx-swift/advanced-patterns.md

Where should my model loading code live?

Is this model shared across features?
├── Yes → Create @Observable service in app/services/
└── No → Is it feature-specific?
    ├── Yes → Create @Observable class in feature/
    └── No → Load inline with @State (simple cases only)

How should I handle conversations?

Foundation Models:
└── Reuse LanguageModelSession for context
    (references/foundation-models/chat-patterns.md #multi-turn)

MLX Swift:
└── Implement custom context management
    (references/mlx-swift/chat-patterns.md)

What generation parameters should I use?

What's the use case?

Factual answers (summaries, facts)
└── temperature: 0.1-0.3

Balanced (chat, Q&A)
└── temperature: 0.6-0.8

Creative (storytelling, ideas)
└── temperature: 0.9-1.2

See references/shared/best-practices.md for details

Resources

• MLX Swift Examples
• Foundation Models Docs
• Hugging Face Model Hub
• MLX-LM Quantization
• MLX Community Models

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: mintuz
• Source: mintuz/skills
• License: MIT

Install and usage instructions live in the source repository linked above.