FastAgent

FastAgent: Simple, Fast, and Strong LLM Agents

[](https://github.com/HKUDS/FastAgent/) [](https://www.python.org/downloads/) [](https://opensource.org/licenses/MIT/) [](./COMMUNICATION.md) [](./COMMUNICATION.md)

🎯 FastAgent's Mission

FastAgent is designed to tackle complex tasks that require both DeepResearch and Computer Use capabilities. While DeepResearch excels at web search, knowledge summarization, and reasoning, and Computer Use focuses on operating complex software applications, many real-world tasks demand both capabilities — such as:

📊 Business Intelligence Reports:

• Research industry trends and competitor insights across multiple data sources.
• Generate automated PowerPoint reports and dynamic dashboards with real-time analytics and visual insights.

📅 Event Planning & Management:

• Research and compare venues and vendors based on cost and requirements.
• Create Excel budgets with cost tracking and manage project schedules.

🛒 Smart Shopping & Price Optimization:

• Compare products, reviews, and prices across e-commerce platforms.
• Automate shopping, apply discounts, and track delivery schedules.

FastAgent bridges this gap by seamlessly integrating intelligent research capabilities with sophisticated computer operation, enabling users to complete end-to-end workflows that span from information gathering and analysis to practical software manipulation—all within a Unified, Simple, and Fast framework.

---

💡 Current Challenges in Multi-Agent Systems

Current agent frameworks face significant challenges when tackling complex, multi-step, real-world tasks:

⚡ Performance Bottlenecks:

• Slow & Unreliable GUI Operations. Complex tasks that humans complete in dozens of steps require hundreds of observe-decide-execute cycles. GUI grounding is particularly slow, error-prone, and frequently freezes when operating real-world interfaces.
• Limited Task Scope. Current approaches constrain agents to narrow, predefined use cases rather than generalizing across diverse scenarios.

❌ High Failure Rates:

• Error Accumulation. Multi-step workflows accumulate cascading errors across execution stages.
• Fragile Cross-Application Transitions. Transitions between heterogeneous software and data sources often cause complete task breakdowns.

🧩 Complex Context Management:

• Overwhelming Multi-Context Load. Multi-faceted tasks generate overwhelming knowledge and tool contexts beyond current systems' capacity.
• Lack of Unified Processing. Systems cannot handle heterogeneous contexts from DeepResearch and Computer Use paradigms, or dynamically integrate diverse user-provided tools, severely limiting extensibility.

🔒 Limited Orchestration & Adaptability:

• Lack of Unified Coordination. Current frameworks lack generalized coordination mechanisms and cannot dynamically adjust to varying task requirements. Each workflow type demands different coordination patterns.
• Manual Workflow Design. Existing systems require complex, task-specific workflow engineering for each scenario, making generalized multi-agent orchestration extremely difficult.

---

🚀 FastAgent's Key Innovations

FastAgent addresses these critical challenges through three Efficient and Effective solutions focusing on Memory Mechanism, Tool Integration, and Multi-Agent Coordination:

🧠 Advanced Memory & Context Management:

Solving Complex Context Handling Challenges

• Adaptive Multi-Tier Memory: Maintains step-, agent-, task- and response-level stores, revealing only the granularity each reasoning hop needs while guarding against knowledge dilution across sprawling workflows
• Intelligent Context Switching: Seamlessly bridges heterogeneous contexts from DeepResearch and Computer Use paradigms, auto-classifying outputs so agents pivot between data mining and GUI actions without losing critical breadcrumbs
• Smart Compression with Dynamic Budgeting: LLM-powered summarizer adaptively compresses based on content purpose: keeps structured data and key decisions for planning tasks, preserves execution details for validation, prunes repetitive operations. Triggers precisely at token thresholds so massive multi-facet contexts never clog the framework
• Incremental Cross-Task Knowledge: Promotes vector embeddings and structured findings to a shared, ever-growing pool updated on every tool call, so future tasks query prior insights instantly and slash redundant computation

🔧 Effortless Tool Integration with Smart Orchestration:

Eliminating Integration Difficulties & Performance Bottlenecks

• Smart Tool RAG System: Precisely retrieves relevant tools from hundreds of available options across all backends (Shell, GUI, MCP, Web), enabling efficient tool selection and reducing context overhead
• Unified Backend Architecture: Provides consistent interface for Shell, GUI, MCP, and Web. Dynamically integrates diverse user-provided tools without manual adaptation, solving extensibility challenges through generalized tool abstraction and lifecycle management
• Zero-Config MCP Support: Plug-and-play MCP server integration without complex setup. Just declare servers in config and FastAgent handles connections, protocol negotiation, and session pooling

🎯 Dynamic Multi-Agent Coordination

Overcoming Limited Orchestration & High Failure Rates

• Event-Driven Kanban Architecture: Provides generalized coordination mechanisms with rule-based workflow routing that dynamically adjusts to varying task requirements. Enables seamless agent communication through shared task states, eliminating manual workflow engineering for every use case
• Selective Quality Assurance: Dedicated EvalAgent activates only when needed (e.g., critical operations, error-prone backends) rather than every step, preventing error accumulation while maintaining efficiency. Ensures reliable transitions between software applications and data sources

---

📋 Table of Contents

• [📊 System Overview](#-system-overview)
• [🎯 Quick Start](#-quick-start)
• [🏗️ Code Structure](#️-code-structure)
• [🔧 Advanced Usage](#-advanced-usage)
• [🔗 Related Projects](#-related-projects)

---

📊 System Overview

FastAgent employs a simple and fast multi-agent event-driven architecture that seamlessly coordinates research and execution capabilities through five core components:

🏗️ Core Architecture

1. 🤝 Multi-Agent Coordination

Dynamic orchestration with intelligent task management

Specialized Agents:

• HostAgent: High-level planning and task decomposition with dependency tracking
• GroundingAgent: Cross-backend execution (Shell, GUI, MCP, Web) with smart tool selection
• EvalAgent: Continuous quality assurance and automatic replanning

Event-Driven Workflow:

• State Management: TODO → IN_PROGRESS → DONE/BLOCKED lifecycle
• Dependency Tracking: Automatic execution ordering based on task dependencies
• Rule-Based Routing: Dynamic agent triggering based on task type and state
• Failure Recovery: Automatic replanning when tasks fail validation

2. 🔧 Unified Tool Integration

Seamless plug-and-play ecosystem with 100+ tools

• Multi-Backend Architecture: Unified interface across Shell, GUI, MCP, and Web
• Smart Tool RAG: Semantic search retrieves relevant tools from hundreds of options
• Zero-Config MCP: Add servers to config file, FastAgent handles discovery and routing
• Session Pooling: Efficient resource management and connection reuse

3. 🧠 Advanced Memory & Context

Intelligent compression and cross-phase awareness

• Multi-Level Storage: Task context, agent history, and processed results
• Smart Compression: Automatically manages overwhelming multi-faceted contexts
• Context Switching: Seamless transitions between research and operation phases
• Historical Awareness: Agents access and learn from previous execution results

4. 🔍 Intelligent Search

Multi-source information retrieval and aggregation

• Web Integration: Built-in search and browsing capabilities
• Tool Discovery: Semantic matching across all available tools and MCP servers
• Knowledge Retrieval: Context-aware search through execution history
• Source Fusion: Combines web, local files, APIs, and databases

5. 🛡️ Security & Control

Enterprise-grade safety with comprehensive audit

• Command Filtering: Whitelist/blacklist with pattern matching
• User Approval: Optional confirmation for sensitive operations
• Access Control: Granular permissions per backend and operation
• Complete Audit: Full logging with trajectory recording and video capture

---

🎯 Quick Start

1. Environment Setup

# Clone repository
git clone https://github.com/HKUDS/FastAgent.git
cd FastAgent

# Create and activate conda environment
conda create -n fastagent python=3.12 -y
conda activate fastagent

# Install dependencies
pip install -r requirements.txt

> [!NOTE] > Create a .env file and add your API keys (refer to fastagent/.env.example).

2. Launch FastAgent

Start Local Server (Required for Computer Control)

The local server is a lightweight Flask service that enables FastAgent to interact with your computer (GUI automation, Python/Bash execution, file operations, screen capture, etc.).

> [!NOTE] > See [fastagent/local_server/README.md](fastagent/local_server/README.md) for complete API documentation and advanced configuration.

> [!IMPORTANT] > Platform-specific setup required: Different operating systems need different dependencies for desktop control. Please install the required dependencies for your OS before starting the local server:

macOS Setup

# Install macOS-specific dependencies
pip install pyobjc-core pyobjc-framework-cocoa pyobjc-framework-quartz atomacos

Permissions Required: macOS will automatically prompt for permissions when you first run the local server. Grant the following:

• Accessibility (for GUI control)
• Screen Recording (for screenshots and video capture)

> If prompts don't appear, manually grant permissions in System Settings → Privacy & Security.

Linux Setup

# Install Linux-specific dependencies
pip install python-xlib pyatspi numpy

# Install system packages
sudo apt install at-spi2-core python3-tk scrot

Windows Setup

# Install Windows-specific dependencies
pip install pywinauto pywin32 PyGetWindow

After installing the platform-specific dependencies, start the local server:

python -m fastagent.local_server.main

> [!TIP] > Local server is required for GUI automation and Python/Bash execution. Without it, only MCP servers and web research capabilities are available.

Start FastAgent

Then, launch the FastAgent main process in another terminal:

python -m fastagent

3. Execute Any Task You Want 🤗

Simply type your task in natural language. FastAgent seamlessly combines DeepResearch and Computer Use to handle complex end-to-end workflows:

> [!TIP] > MCP Server Configuration: For tasks requiring specific tools, add relevant MCP servers to fastagent/config/config_mcp.json. Unsure which servers to add? Simply add all potentially useful ones, FastAgent's Smart Tool RAG will automatically select the appropriate tools for your task. See [MCP Configuration](#mcp-configuration) for details.

Simple Example:

>>> Please help me search HKUDS on Google.

Complex Example - AI Coding Assistants Competitive Analysis

>>> Create a competitive analysis report for AI coding assistants.

Research these 3 products: GitHub Copilot, Cursor, Claude Code.

For each product, find and verify:
1. Supported programming languages and IDE integrations
2. Pricing tiers (in USD) and token limits
3. Key differentiating features (refactoring, test generation, chat capabilities)
4. Security and privacy guarantees

Then create an Excel workbook named "AI_Coding_Assistants_Analysis.xlsx" with:
- Sheet 1: Feature comparison matrix (products as rows, capabilities as columns)
- Sheet 2: Pricing breakdown with cost analysis

Also create a "executive_summary.md" file highlighting the best choice and why.

---

🏗️ Code Structure

📖 Quick Overview

> Legend: ⚡ Core modules | 🔧 Supporting modules

FastAgent/
├── fastagent/
│   ├── __init__.py                       # Package exports
│   ├── __main__.py                       # CLI entry point
│   ├── fastagent.py                      # Main FastAgent class
│   │
│   ├── ⚡ agents/                         # Multi-Agent System 
│   ├── ⚡ workflow/                       # Event-Driven Workflow 
│   ├── ⚡ kanban/                         # Task Management System 
│   ├── ⚡ grounding/                      # Unified Backend System
│   │   ├── core/                         # Core abstractions
│   │   └── backends/                     # Backend implementations
│   │       ├── shell/                    # Shell command execution
│   │       ├── gui/                      # Anthropic Computer Use
│   │       ├── mcp/                      # Model Context Protocol
│   │       └── web/                      # Web search & browsing
│   │
│   ├── ⚡ memory/                         # Memory & Storage
│   ├── ⚡ prompts/                        # Centralized Agent Prompts
│   ├── 🔧 llm/                           # LLM Integration
│   ├── 🔧 config/                        # Configuration System 
│   ├── 🔧 local_server/                  # GUI Backend Server 
│   ├── 🔧 recording/                     # Execution Recording
│   ├── 🔧 platform/                      # Platform Integration
│   └── 🔧 utils/                         # Utilities
│
├── .fastagent/                           # Runtime cache
│   └── embedding_cache/                  # Tool embeddings for Smart Tool RAG
│
├── logs/                                 # Execution logs and recordings
│   ├── __main__/                         # Application logs
│   └── recordings/                       # Complete execution audit trail
│
├── requirements.txt                      # Python dependencies
└── README.md                            

---

📂 Detailed Module Structure

⚡ agents/ - Multi-Agent System (Core Architecture)

agents/
├── __init__.py
├── base.py                         # Base agent class with common functionality
├── coordinator.py                  # Agent coordination & resource management
├── host_agent.py                   # High-level planning and task decomposition
├── grounding_agent.py              # Cross-backend task execution
├── eval_agent.py                   # Automatic evaluation and quality assurance
├── content_processor.py            # Intelligent content processing
└── agent_data_manager.py           # Agent data storage and retrieval

Key Responsibilities: Task planning, execution, evaluation, and inter-agent coordination.

⚡ workflow/ - Event-Driven Workflow Engine

workflow/
├── __init__.py
├── engine.py                       # Event-driven workflow orchestration
├── rules.py                        # Workflow rule definitions and routing
└── context_manager.py              # Cross-execution context management

Key Responsibilities: Event processing, rule-based routing, state transitions, and context preservation.

⚡ kanban/ - Task Management System

kanban/
├── __init__.py
├── kanban.py                       # Task board and state management
└── enums.py                        # Task types, states, and events

Key Responsibilities: Task lifecycle management, dependency tracking, and state transitions (TODO → IN_PROGRESS → DONE/BLOCKED).

⚡ grounding/ - Unified Backend System (Core Integration Layer)

Core Abstractions

grounding/core/
├── grounding_client.py

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [HKUDS](https://github.com/HKUDS)
- **Source:** [HKUDS/FastAgent](https://github.com/HKUDS/FastAgent)
- **License:** MIT

Install and usage instructions live in the source repository linked above.