# Dspy Adapters Multimodal

> Use for DSPy adapter selection, JSONAdapter, XMLAdapter, ChatAdapter, native function calling, structured outputs, and multimodal inputs like dspy.Image or dspy.Audio.

- **Type:** Skill
- **Install:** `agentstack add skill-omidzamani-dspy-skills-dspy-adapters-multimodal`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [OmidZamani](https://agentstack.voostack.com/s/omidzamani)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [OmidZamani](https://github.com/OmidZamani)
- **Source:** https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-adapters-multimodal

## Install

```sh
agentstack add skill-omidzamani-dspy-skills-dspy-adapters-multimodal
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# DSPy Adapters and Multimodal I/O

## Goal

Choose an adapter deliberately and model image, audio, and file inputs with DSPy's typed primitives.

## Adapter Selection

| Adapter | Use it for |
|---------|------------|
| `dspy.ChatAdapter()` | Default, human-readable field markers, broad model compatibility |
| `dspy.JSONAdapter()` | Structured JSON output and native function calling where supported |
| `dspy.XMLAdapter()` | XML-tagged fields when XML is easier for the target LM to follow |
| `dspy.TwoStepAdapter()` | A separate extraction pass when parsing needs extra help |

Configure globally or for a limited scope:

```python
import dspy

dspy.configure(
    lm=dspy.LM("openai/gpt-4o-mini"),
    adapter=dspy.JSONAdapter(),
)

with dspy.context(adapter=dspy.XMLAdapter()):
    result = dspy.Predict("question -> answer")(question="What is DSPy?")
```

## Native Function Calling

`JSONAdapter` enables native function calling by default. `ChatAdapter` keeps text parsing by default. Override either behavior explicitly:

```python
chat_native = dspy.ChatAdapter(use_native_function_calling=True)
json_manual = dspy.JSONAdapter(use_native_function_calling=False)
```

DSPy falls back to manual parsing when the configured LM does not support native function calling.

## Image Inputs

```python
class DescribeImage(dspy.Signature):
    image: dspy.Image = dspy.InputField()
    description: str = dspy.OutputField()

describe = dspy.Predict(DescribeImage)
result = describe(image=dspy.Image("./diagram.png"))
```

Pass a local path, HTTP URL, bytes, PIL image, or existing data URI directly to `dspy.Image(...)`.

## Audio and File Inputs

```python
class SummarizeAudio(dspy.Signature):
    audio: dspy.Audio = dspy.InputField()
    summary: str = dspy.OutputField()

audio = dspy.Audio.from_file("./meeting.wav")
summary = dspy.Predict(SummarizeAudio)(audio=audio)
```

```python
class SummarizeFile(dspy.Signature):
    file: dspy.File = dspy.InputField()
    summary: str = dspy.OutputField()

document = dspy.File.from_path("./research.pdf")
summary = dspy.Predict(SummarizeFile)(file=document)
```

Provider capabilities vary. Verify that the selected model accepts the media type before deployment.

## Best Practices

1. Start with `ChatAdapter`; switch only for a measured reason.
2. Use typed signatures for structured output.
3. Test adapter behavior against the exact production model.
4. Avoid deprecated `Image.from_file()` and `Image.from_url()` helpers; call `dspy.Image(...)`.
5. Keep local file handling and uploaded file IDs within provider policy.

## Related Skills

- Design signatures: [dspy-signature-designer](../dspy-signature-designer/SKILL.md)
- Build tool agents: [dspy-react-agent-builder](../dspy-react-agent-builder/SKILL.md)

## Official Documentation

- **Adapters guide**: https://dspy.ai/learn/programming/adapters/
- **Tools guide**: https://dspy.ai/learn/programming/tools/
- **XMLAdapter API**: https://dspy.ai/api/adapters/XMLAdapter/
- **Image API**: https://dspy.ai/api/primitives/Image/
- **Audio API**: https://dspy.ai/api/primitives/Audio/

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [OmidZamani](https://github.com/OmidZamani)
- **Source:** [OmidZamani/dspy-skills](https://github.com/OmidZamani/dspy-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-omidzamani-dspy-skills-dspy-adapters-multimodal
- Seller: https://agentstack.voostack.com/s/omidzamani
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.