# Fastdeploy Deploy Simple

> Install FastDeploy and deploy an OpenAI-compatible LLM/VLM serving endpoint on NVIDIA GPU, Kunlunxin XPU, Hygon DCU, or other supported hardware.

- **Type:** Skill
- **Install:** `agentstack add skill-yuanlehome-fastdeploy-skills-fastdeploy-deploy-simple`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [yuanlehome](https://agentstack.voostack.com/s/yuanlehome)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [yuanlehome](https://github.com/yuanlehome)
- **Source:** https://github.com/yuanlehome/fastdeploy-skills/tree/master/skills/fastdeploy-deploy-simple

## Install

```sh
agentstack add skill-yuanlehome-fastdeploy-skills-fastdeploy-deploy-simple
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# FastDeploy Simple Deployment

A skill to install FastDeploy, start an OpenAI-compatible server, and validate the API — all in a few commands.

## What this skill does

- Detect hardware backend (NVIDIA CUDA SM80/90, NVIDIA CUDA SM86/89, Kunlunxin XPU, Hygon DCU, or CPU)
- Install PaddlePaddle and FastDeploy with the appropriate backend packages from PaddlePaddle's package index
- Start the FastDeploy server via `python -m fastdeploy.entrypoints.openai.api_server`
- Test the OpenAI-compatible API (`/v1/chat/completions`)
- Support virtual environment isolation
- Support streaming and non-streaming responses

## Prerequisites

- OS: Linux (X86_64)
- Python: 3.10 – 3.12
- GPU Driver ≥ 535, CUDA ≥ 12.3, cuDNN ≥ 9.5 (NVIDIA GPU path)
- pip package manager (uv also supported)
- curl (for API testing)

> **Important:** FastDeploy packages are **NOT available on PyPI**. You must install from PaddlePaddle's dedicated package index as shown below.

## Installation

### NVIDIA GPU (SM80/90 — A30/A100/H100/A800/H800)

```bash
# Step 1: Install PaddlePaddle GPU
python -m pip install paddlepaddle-gpu==3.3.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/cu126/

# Step 2: Install FastDeploy GPU (SM80/90 index)
python -m pip install fastdeploy-gpu==2.4.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/fastdeploy-gpu-80_90/ \
    --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
```

### NVIDIA GPU (SM86/89 — A10/L20/L40/RTX 4090)

```bash
# Step 1: Install PaddlePaddle GPU
python -m pip install paddlepaddle-gpu==3.3.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/cu126/

# Step 2: Install FastDeploy GPU (SM86/89 index)
python -m pip install fastdeploy-gpu==2.4.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/fastdeploy-gpu-86_89/ \
    --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
```

> **Note:** The pre-built Docker image only supports SM80/90 GPUs. If deploying on SM86/89 GPUs (L40/4090/L20), reinstall `fastdeploy-gpu` after creating the container using the SM86/89 index above.

### Kunlunxin XPU (P800)

```bash
# Step 1: Install PaddlePaddle XPU
python -m pip install paddlepaddle-xpu==3.3.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/xpu-p800/

# Step 2: Install FastDeploy XPU
python -m pip install fastdeploy-xpu==2.4.0 \
    -i https://www.paddlepaddle.org.cn/packages/stable/fastdeploy-xpu-p800/ \
    --extra-index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
```

### Verify installation

```bash
python -c "import fastdeploy; print(fastdeploy.__version__)"
```

## Usage

### Create a virtual environment (optional but recommended)

```bash
# Using uv (recommended)
uv venv .venv --python 3.12
source .venv/bin/activate

# Or using standard venv
python3 -m venv .venv
source .venv/bin/activate
```

### Run the complete workflow (suggested)

Use `scripts/quickstart.sh` with default options (model: `baidu/ERNIE-4.5-0.3B-Paddle`, port: 8180):

```bash
scripts/quickstart.sh
```

Or with custom options:

```bash
# Specify model and port
scripts/quickstart.sh --model "baidu/ERNIE-4.5-0.3B-Paddle" --port 8180

# With GPU memory utilization
scripts/quickstart.sh --model "baidu/ERNIE-4.5-21B-A3B-Paddle" --port 8180 --gpu_memory_utilization 0.8

# With virtual environment
scripts/quickstart.sh --venv /path/to/venv --model "baidu/ERNIE-4.5-0.3B-Paddle" --port 8180
```

### Start server manually

```bash
export ENABLE_V1_KVCACHE_SCHEDULER=1
python -m fastdeploy.entrypoints.openai.api_server \
    --model baidu/ERNIE-4.5-0.3B-Paddle \
    --port 8180 \
    --metrics-port 8181 \
    --engine-worker-queue-port 8182 \
    --max-model-len 32768 \
    --max-num-seqs 32
```

Key parameters:
- `--model`: Model name or local path. If not found locally, auto-downloads from AIStudio.
- `--port`: HTTP service port (default 8000; quick start guide uses 8180)
- `--metrics-port`: Prometheus metrics endpoint port
- `--max-model-len`: Maximum context length in tokens (default: 2048)
- `--max-num-seqs`: Maximum concurrent requests in Decode phase (default: 8)
- `--tensor-parallel-size`: Number of GPUs for tensor parallelism (default: 1)
- `--gpu-memory-utilization`: GPU memory utilization ratio (default: 0.9)
- `--enable-prefix-caching`: Enable prefix caching to speed up repeated prompts
- `--quantization`: Online quantization: `wint4`, `wint8`, `block_wise_fp8`, `wfp8afp8`

> **Note:** `--max-model-len` and `--max-num-seqs` should be tuned based on your GPU memory. See [Parameter Documentation](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/parameters.md) for guidance.

### Run individual script commands (step-by-step)

```bash
# Install only
scripts/quickstart.sh install

# Start server
scripts/quickstart.sh start --model "baidu/ERNIE-4.5-0.3B-Paddle" --port 8180

# Test API
scripts/quickstart.sh test --port 8180

# Check status
scripts/quickstart.sh status

# Stop server
scripts/quickstart.sh stop

# Restart server
scripts/quickstart.sh restart --port 8180
```

## Configuration

```
scripts/quickstart.sh [command] [OPTIONS]

Commands:
  install  - Install PaddlePaddle + FastDeploy for detected hardware
  start    - Start the FastDeploy OpenAI-compatible server
  stop     - Stop the FastDeploy server
  test     - Test the OpenAI-compatible API
  status   - Show server status and recent logs
  restart  - Stop then start the server
  all      - Run complete workflow (default)

Options:
  --model MODEL                  Model name or local path
                                 (default: baidu/ERNIE-4.5-0.3B-Paddle)
  --port PORT                    Server port (default: 8180)
  --venv VENV_PATH               Virtual environment path (optional)
  --gpu_memory_utilization VRAM  GPU memory utilization ratio (default: 0.8)
  --max_model_len N              Max context length in tokens (default: 32768)
  --max_num_seqs N               Max concurrent requests (default: 32)
```

### Hardware Backend Detection

The script automatically detects your hardware:

| Hardware | Detection Method | Package |
|---|---|---|
| NVIDIA CUDA SM80/90 | `nvidia-smi`, compute_cap ≤ 8.0 or ≥ 9.0 | `fastdeploy-gpu` (80_90 index) |
| NVIDIA CUDA SM86/89 | `nvidia-smi`, compute_cap 8.6–8.9 | `fastdeploy-gpu` (86_89 index) |
| Kunlunxin XPU | `xpu_smi` or `KUNLUN_VISIBLE_DEVICES` | `fastdeploy-xpu` |
| Hygon DCU | `/dev/kfd` + `/dev/dri` devices | ROCm-compatible build |
| CPU | Fallback | `fastdeploy` |

## API Testing

After the server starts successfully, you should see:

```
api_server.py[line:91] Launching metrics service at http://0.0.0.0:8181/metrics
api_server.py[line:94] Launching chat completion service at http://0.0.0.0:8180/v1/chat/completions
INFO:     Uvicorn running on http://0.0.0.0:8180 (Press CTRL+C to quit)
```

### Health check

```bash
curl -i http://localhost:8180/health
```

### Chat completion (curl)

```bash
curl -X POST "http://localhost:8180/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Say hello!"}],
    "max_tokens": 50,
    "stream": false
  }'
```

### Streaming response

```bash
curl -X POST "http://localhost:8180/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Write a short poem."}],
    "stream": true
  }'
```

### Python client (OpenAI-compatible)

```python
import openai

client = openai.Client(
    base_url="http://localhost:8180/v1",
    api_key="null"
)

response = client.chat.completions.create(
    model="null",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Write me a poem about large language models."},
    ],
    stream=True,
)
for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
print()
```

## Supported Models

FastDeploy supports auto-downloading models from AIStudio (default), ModelScope, or HuggingFace:

```bash
# Set download source (AISTUDIO / MODELSCOPE / HUGGINGFACE)
export FD_MODEL_SOURCE=AISTUDIO
# Set download cache directory
export FD_MODEL_CACHE=/ssd1/models
```

### Large Language Models (LLM)

| Model | Precisions | Example Model ID |
|---|---|---|
| ERNIE-4.5-0.3B | BF16 | `baidu/ERNIE-4.5-0.3B-Paddle` |
| ERNIE-4.5-21B-A3B | BF16, WINT4, WINT8 | `baidu/ERNIE-4.5-21B-A3B-Paddle` |
| ERNIE-4.5-21B-A3B-Thinking | BF16 | `baidu/ERNIE-4.5-21B-A3B-Thinking` |
| ERNIE-4.5-300B-A47B | BF16, WINT4, WINT8, FP8 | `baidu/ERNIE-4.5-300B-A47B-Paddle` |
| Qwen3 | BF16, WINT8, FP8 | `Qwen/qwen3-8B` |
| Qwen3-MoE | BF16, WINT4, FP8 | `Qwen/Qwen3-30B-A3B` |
| Qwen2.5 | BF16, WINT8, FP8 | `Qwen/qwen2.5-7B` |
| DeepSeek-V3/R1 | BF16, WINT4 | `unsloth/DeepSeek-V3-0324-BF16` |
| GLM-4.5/4.6 | BF16, wfp8afp8 | `zai-org/GLM-4.5-Air` |

### Multimodal Language Models (VLM)

| Model | Precisions | Example Model ID |
|---|---|---|
| ERNIE-4.5-VL-28B-A3B | BF16, WINT4, WINT8 | `baidu/ERNIE-4.5-VL-28B-A3B-Paddle` |
| ERNIE-4.5-VL-28B-A3B-Thinking | BF16 | `baidu/ERNIE-4.5-VL-28B-A3B-Thinking` |
| ERNIE-4.5-VL-424B-A47B | BF16, WINT4, WINT8 | `baidu/ERNIE-4.5-VL-424B-A47B-Paddle` |
| PaddleOCR-VL | BF16, WINT4, WINT8 | `PaddlePaddle/PaddleOCR-VL` |
| Qwen2.5-VL | BF16, WINT4, FP8 | `Qwen/Qwen2.5-VL-7B-Instruct` |

See [full supported models list](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/supported_models.md).

## Troubleshooting

**`fastdeploy` not found / import error:**
- Ensure you installed from the PaddlePaddle index, NOT PyPI
- Check: `python -c "import fastdeploy; print(fastdeploy.__version__)"`

**Server won't start:**
- Check port conflicts: `lsof -i :8180`
- Verify GPU: `nvidia-smi` (NVIDIA) or `xpu_smi` (Kunlunxin)
- Check logs: `cat ./tmp/fastdeploy-server.log`

**Model download fails:**
- Set `FD_MODEL_SOURCE` to `MODELSCOPE` or `HUGGINGFACE` if AIStudio is unreachable
- For HuggingFace private models: set `HF_TOKEN` environment variable

**Out of memory (OOM):**
- Use a smaller model (e.g., `baidu/ERNIE-4.5-0.3B-Paddle`)
- Reduce `--gpu-memory-utilization` (e.g., `0.7`)
- Reduce `--max-num-seqs` or `--max-model-len`

**Wrong backend detected:**
- NVIDIA: ensure `nvidia-smi` is in PATH
- Kunlunxin: set `KUNLUN_VISIBLE_DEVICES` or ensure `xpu_smi` is in PATH

## Notes

- Server logs: `./tmp/fastdeploy-server.log` (or `$VENV_PATH/tmp/fastdeploy-server.log`)
- Server PID: `./tmp/fastdeploy-server.pid`
- First run downloads model weights automatically; subsequent runs use cached weights
- `ENABLE_V1_KVCACHE_SCHEDULER=1` enables the V1 KV cache scheduler (recommended)
- Add `--enable-prefix-caching` and `--swap-space 50` (GB) to enable prefix caching

## References

- [FastDeploy Quick Start (10-Min Deploy)](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/get_started/quick_start.md)
- [NVIDIA GPU Installation Guide](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/get_started/installation/nvidia_gpu.md)
- [Kunlunxin XPU Installation Guide](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/get_started/installation/kunlunxin_xpu.md)
- [Online Serving Documentation](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/online_serving/README.md)
- [Full Parameter Reference](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/parameters.md)
- [Supported Models List](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/supported_models.md)
- [FastDeploy Documentation Site](https://paddlepaddle.github.io/FastDeploy/)

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [yuanlehome](https://github.com/yuanlehome)
- **Source:** [yuanlehome/fastdeploy-skills](https://github.com/yuanlehome/fastdeploy-skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-yuanlehome-fastdeploy-skills-fastdeploy-deploy-simple
- Seller: https://agentstack.voostack.com/s/yuanlehome
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.