Fastdeploy Offline Inference
Run FastDeploy offline batch inference using the LLM Python API — no server required. Covers LLM.chat and LLM.generate interfaces, SamplingParams, multimodal inputs, and reasoning models.
Fastdeploy Deploy Simple
Install FastDeploy and deploy an OpenAI-compatible LLM/VLM serving endpoint on NVIDIA GPU, Kunlunxin XPU, Hygon DCU, or other supported hardware.
Fastdeploy Deploy Docker
Deploy FastDeploy using the official pre-built Docker image or build from source. Supports NVIDIA GPU (SM80/90 and SM86/89), Kunlunxin XPU, and other hardware. Runs the OpenAI-compatible server with GPU access.
Fastdeploy Advanced Features
Advanced FastDeploy features — PD disaggregated deployment, speculative decoding (MTP/Ngram/Suffix), quantization (WINT4/WINT8/FP8), prefix caching, chunked prefill, and the load-balancing Router. Based on FastDeploy v2.4 documentation.