Ask The Web 103

Project 3 - Build an "Ask-the-Web" Agent similar to Perplexity with Tool calling

A production-grade, Perplexity-like AI research agent

built with ReACT · ReWOO · Reflexion · Tree Search · MCP · A2A

[](https://python.org) [](https://fastapi.tiangolo.com) [](https://openai.com) [](https://anthropic.com) [](https://redis.io) [](https://docker.com) [](LICENSE) [](https://pytest.org)

> Ask anything. The agent searches, reasons, verifies, and answers — > with full citations, streaming output, and production-grade reliability. > ### To better understand this project, first visit this link for a visualization of the project and what I built: Link > ### Then, if you want to learn each topic in a tutorial format, read this file thoroughly: Link

[Quick Start](#-quick-start) • [Architecture](#-architecture) • [Agents](#-agent-types) • [API Reference](#-api-reference) • [Configuration](#-configuration) • [Evaluation](#-evaluation) • [Contributing](#-contributing)

---

Table of Contents

• [What Is This?](#-what-is-this)
• [Key Features](#-key-features)
• [Architecture](#-architecture)
• [Project Structure](#-project-structure)
• [Quick Start](#-quick-start)
• [Prerequisites](#prerequisites)
• [Installation](#installation)
• [Environment Setup](#environment-setup)
• [Running Locally](#running-locally)
• [Running with Docker](#running-with-docker)
• [Agent Types](#-agent-types)
• [ReACT Agent](#1-react-agent)
• [Reflexion Agent](#2-reflexion-agent)
• [ReWOO Agent](#3-rewoo-agent)
• [Orchestrator Agent](#4-orchestrator-agent)
• [Tree Search Agent](#5-tree-search-agent)
• [Workflows](#-workflows)
• [Prompt Chaining](#prompt-chaining)
• [Routing](#routing)
• [Parallelization](#parallelization)
• [Reflection](#reflection)
• [Tools](#-tools)
• [Built-in Tools](#built-in-tools)
• [MCP Integration](#mcp-integration)
• [Adding Custom Tools](#adding-custom-tools)
• [Multi-Agent Systems](#-multi-agent-systems)
• [Orchestrator-Worker](#orchestrator-worker-pattern)
• [A2A Protocol](#a2a-agent-to-agent-protocol)
• [API Reference](#-api-reference)
• [Endpoints](#endpoints)
• [Request & Response Schemas](#request--response-schemas)
• [Streaming (SSE)](#streaming-sse)
• [Authentication](#authentication)
• [Configuration](#-configuration)
• [Evaluation](#-evaluation)
• [Answer Quality Metrics](#answer-quality-metrics)
• [Running Benchmarks](#running-benchmarks)
• [Observability](#-observability)
• [Testing](#-testing)
• [Deployment](#-deployment)
• [Roadmap](#-roadmap)
• [Contributing](#-contributing)
• [License](#-license)

---

What Is This?

Ask-the-Web Agent is a production-ready AI research assistant that works like Perplexity AI — but fully open, self-hosted, and extensible.

You ask a question in natural language. The agent:

1. Plans how to answer it (which strategy, how many steps)
2. Searches the web in real time using Tavily or SerpAPI
3. Scrapes relevant pages for detailed content
4. Reasons step-by-step using one of five agent strategies
5. Verifies its own answer through self-critique (Reflexion)
6. Synthesizes a final, cited, markdown-formatted answer
7. Streams the result token-by-token to the client

Unlike a raw LLM, this agent never makes up facts — every claim is grounded in real-time web sources with inline citations.

Why build this?

| Problem with raw LLMs | How this agent solves it | |---|---| | Knowledge cutoff (training data is stale) | Real-time web search on every query | | Hallucination (confident but wrong) | Source-grounded answers + Reflexion critique | | No citations (can't verify claims) | Every fact linked to a URL | | Single-shot (one chance to get it right) | Multi-step reasoning with tool loops | | Can't handle complex multi-part questions | Orchestrator decomposes and parallelizes |

---

Key Features

Five Agent Strategies

Choose automatically via smart routing or manually per request:

• ReACT — Fast, iterative reason-and-act loops
• Reflexion — ReACT + self-critique and automatic revision
• ReWOO — Full plan upfront, parallel execution, single synthesis
• Orchestrator — Decomposes complex queries into parallel sub-agents
• Tree Search — Explores multiple reasoning paths, picks the best

Production Tool Stack

• Web Search — Tavily (primary) or SerpAPI (fallback)
• Web Scraper — Playwright + BeautifulSoup, cleans boilerplate
• Calculator — Safe sandboxed math expression evaluator
• Summarizer — Condenses long scraped content
• MCP Support — Connect any Model Context Protocol server

Multi-Agent Coordination

• Orchestrator-Worker — Spawn N parallel specialist agents
• A2A Protocol — Agent-to-Agent HTTP communication standard
• MultiAgentCoordinator — Route tasks to registered specialist agents

API & Streaming

• REST API — FastAPI with full OpenAPI docs
• SSE Streaming — Token-by-token answer delivery
• Redis Cache — SHA256-keyed response caching (1hr TTL)
• Rate Limiting — Per-IP sliding window

Evaluation System

• LLM-as-Judge — Multi-dimensional answer quality scoring
• Text Metrics — Citation coverage, structure, length (no LLM cost)
• Benchmark Suite — 5 built-in test cases across categories
• Parallel Voting — Majority-vote answer verification

Production Infrastructure

• Structured logging — structlog + rich, JSON in production
• Prometheus metrics — /metrics endpoint
• Docker + Compose — One-command deployment
• Retry logic — Tenacity-backed exponential backoff
• Context management — Automatic token trimming at window limits
• Multi-provider — Switch between OpenAI and Anthropic

---

Architecture

System Overview

                        ┌─────────────────────────────────┐
                        │         Client (HTTP/SSE)        │
                        └──────────────┬──────────────────┘
                                       │
                        ┌──────────────▼──────────────────┐
                        │         FastAPI (REST API)       │
                        │  middleware: rate limit, logging  │
                        │  middleware: request ID, errors   │
                        └──────────────┬──────────────────┘
                                       │
                        ┌──────────────▼──────────────────┐
                        │          Redis Cache             │
                        │   (SHA256 keyed, 1hr TTL)        │
                        └──────────────┬──────────────────┘
                                  miss │
                        ┌─────────────▼───────────────────┐
                        │         Query Router             │
                        │  rule-based pre-filter +         │
                        │  LLM-based classification        │
                        └──┬───────┬──────┬──────┬────────┘
                           │       │      │      │
              ┌────────────▼─┐ ┌───▼──┐ ┌▼────┐ ┌▼──────────────┐
              │  ReACT Agent │ │ReWOO │ │Refl.│ │  Orchestrator │
              │  (fast Q&A)  │ │Agent │ │Agent│ │  (multi-part) │
              └──────┬───────┘ └──┬───┘ └──┬──┘ └──────┬────────┘
                     │            │         │            │
              ┌──────▼────────────▼─────────▼────────────▼───────┐
              │                 Tool Executor                      │
              │          (parallel or sequential)                  │
              └───┬──────────┬──────────┬──────────┬─────────────┘
                  │          │          │          │
           ┌──────▼──┐ ┌─────▼───┐ ┌───▼────┐ ┌──▼──────────┐
           │   Web   │ │  Web    │ │ Calc-  │ │     MCP     │
           │ Search  │ │ Scraper │ │ ulator │ │   Servers   │
           └─────────┘ └─────────┘ └────────┘ └─────────────┘

Agent Decision Flow

User Query
    │
    ▼
┌───────────────────────────────────────────────┐
│              TaskPlanner                       │
│   Analyzes complexity → PlanningLevel (1-5)   │
└───────────────────────┬───────────────────────┘
                        │
          ┌─────────────▼──────────────┐
          │       QueryRouter          │
          │  Rule-based quick classify │
          │  ──────────────────────── │
          │  LLM-based deep classify   │
          └─────┬──────┬──────┬───────┘
                │      │      │
     ┌──────────▼─┐  ┌─▼────┐ ┌▼───────────────────┐
     │  simple_qa │  │ calc │ │  research /         │
     │  → ReACT   │  │→ReACT│ │  multi_faceted /    │
     └────────────┘  └──────┘ │  → Reflexion /      │
                               │  → Orchestrator     │
                               └─────────────────────┘
                                         │
                               ┌─────────▼──────────┐
                               │   ReACT Loop        │
                               │   ┌─────────────┐   │
                               │   │   THINK     │   │
                               │   │  (LLM call) │   │
                               │   └──────┬──────┘   │
                               │          │           │
                               │   ┌──────▼──────┐   │
                               │   │     ACT     │   │
                               │   │ (tool calls)│   │
                               │   └──────┬──────┘   │
                               │          │           │
                               │   ┌──────▼──────┐   │
                               │   │   OBSERVE   │   │
                               │   │  (results)  │   │
                               │   └──────┬──────┘   │
                               │          │           │
                               │     done?│ no → loop │
                               └──────────┼───────────┘
                                          │ yes
                               ┌──────────▼───────────┐
                               │    Final Answer       │
                               │  (with citations)     │
                               └──────────────────────┘

Token & Context Management

Every LLM call:
    messages → TokenCounter.count_messages()
                        │
              exceeds context limit?
                   yes │          no
                        │           │
          trim_to_fit() │           │ → proceed
          (drop oldest  │
           non-system   │
           messages)    │
                        └──────────►│ → LLM call

---

📁 Project Structure

ask_the_web_agent/
│
├── 📄 pyproject.toml              # Dependencies, build config, tool settings
├── 📄 .env.example                # All environment variables documented
├── 📄 docker-compose.yml          # Agent + Redis + Prometheus
├── 📄 Dockerfile                  # Multi-stage build (builder + runtime)
├── 📄 README.md                   # This file
│
├── 📁 configs/                    # Application configuration
│   ├── settings.py                # Pydantic Settings (type-safe env loading)
│   ├── logging_config.py          # structlog + rich setup
│   └── prometheus.yml             # Prometheus scrape config
│
├── 📁 core/                       # Shared infrastructure
│   ├── exceptions.py              # Full exception hierarchy
│   ├── message_types.py           # Message, ToolCall, AgentState types
│   ├── token_counter.py           # tiktoken-based counter + trim
│   └── llm_client.py              # Unified OpenAI + Anthropic client
│
├── 📁 tools/                      # Tool layer
│   ├── base_tool.py               # BaseTool ABC + ToolDefinition schema
│   ├── tool_registry.py           # Central tool store
│   ├── tool_executor.py           # Parallel + sequential execution
│   ├── web_search.py              # Tavily / SerpAPI search
│   ├── web_scraper.py             # httpx + BeautifulSoup scraper
│   ├── calculator.py              # Safe sandboxed math eval
│   ├── summarizer.py              # Extractive text summarizer
│   └── mcp_client.py              # MCP protocol client + registry
│
├── 📁 agents/                     # Agent implementations
│   ├── base_agent.py              # Abstract base + shared utilities
│   ├── react_agent.py             # ReACT: iterative reason-act-observe
│   ├── reflexion_agent.py         # Reflexion: ReACT + self-critique
│   ├── rewoo_agent.py             # ReWOO: plan-execute-solve
│   ├── orchestrator.py            # Orchestrator-Worker: decompose + parallel
│   ├── tree_search_agent.py       # Best-first tree search
│   ├── planner.py                 # Task planner + PlanningLevel
│   └── a2a.py                     # Agent-to-Agent protocol
│
├── 📁 workflows/                  # Workflow patterns
│   ├── prompt_chaining.py         # Sequential chained LLM calls
│   ├── routing.py                 # LLM + rule-based query router
│   ├── parallelization.py         # Sectioning + voting patterns
│   ├── reflection.py              # Standalone critique-revise loop
│   └── __init__.py                # build_routed_pipeline()
│
├── 📁 evaluation/                 # Quality assessment
│   ├── metrics.py                 # Fast rule-based text metrics
│   ├── evaluator.py               # LLM-as-judge evaluator
│   └── benchmarks.py              # Benchmark runner + built-in cases
│
├── 📁 api/                        # FastAPI application
│   ├── main.py                    # App factory + lifespan
│   ├── routes.py                  # All endpoint handlers
│   ├── schemas.py                 # Pydantic request/response models
│   ├── middleware.py              # Rate limit, logging, error handling
│   └── cache.py                   # Redis response cache
│
└── 📁 tests/                      # Full test suite
    ├── test_tools.py              # Tool unit tests
    ├── test_agents.py             # Agent behavior tests
    ├── test_workflows.py          # Workflow + metric tests
    └── test_api.py                # API endpoint + middleware tests

---

Quick Start

Prerequisites

| Requirement | Version | Notes | |---|---|---| | Python | 3.11+ | Uses match statements, Self type | | Redis | 7+ | For response caching | | Docker | 24+ | Optional, for containerized run | | OpenAI API Key | — | Primary LLM provider | | Tavily API Key | — | Primary search provider |

> Minimum to get started: Python 3.11 + OpenAI key + Tavily key. > Redis and Docker are optional for local development.

---

Installation

Option A — pip (development)

# 1. Clone the repository
git https://github.com/AdilShamim8/Ask-the-Web-103.git
cd ask-the-web-agent

# 2. Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# 3. Install with all dev dependencies
pip install -e ".[dev]"

# 4. Install Playwright browser (for web scraping)
playwright install chromium

# 5. Verify installation
python -c "import openai, fastapi, redis; print('✅ All dependencies OK')"

Option B — Docker (production)

git clone https://github.com/AdilShamim8/Ask-the-Web-103.git
cd ask-the-web-agent
cp .env.example .env
# Edit .env with your API keys
docker-compose up -d

---

Environment Setup

Copy the example and fill in your keys:

cp .env.example .env

Open .env and set the required values:

# ── REQUIRED ─────────

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [AdilShamim8](https://github.com/AdilShamim8)
- **Source:** [AdilShamim8/Ask-the-Web-103](https://github.com/AdilShamim8/Ask-the-Web-103)
- **License:** MIT
- **Homepage:** https://ask-the-web.space-z.ai/

Install and usage instructions live in the source repository linked above.