RAG Workflow Planner

RAG Workflow Planner

What this skill does

This skill walks through your RAG use case requirements and designs a complete, production-ready Retrieval-Augmented Generation pipeline. It recommends a chunking strategy, embedding model, vector store, retrieval approach, and reranking strategy — each choice justified against your specific requirements. The output includes an ASCII architecture diagram, a component-by-component breakdown, and a starter code outline.

Use this when you're starting a new RAG project or when an existing RAG pipeline is producing poor retrieval quality and you want a systematic redesign.

How to use

Claude Code / Cline

Copy this file to .agents/skills/rag-workflow-planner/SKILL.md in your project root.

Then describe your use case and ask:

• "Use the RAG Workflow Planner skill to design a pipeline for our customer support chatbot."
• "Plan a RAG architecture for our internal knowledge base search using the RAG Workflow Planner skill."

The more detail you provide about document types, query patterns, and latency requirements, the more precise the output.

Cursor

Add the "Prompt / Instructions" section to your .cursorrules file. Describe your RAG use case in the chat.

Codex

Describe your use case and answer the requirements questions in the instructions below, then include the full instructions. Codex will produce the architecture design.

The Prompt / Instructions for the Agent

When asked to design a RAG pipeline, follow these steps:

1. Gather requirements. If not provided, ask for:

• Document types: PDFs, HTML pages, markdown files, database records, code files, emails, etc.
• Corpus size: A few dozen documents, thousands, millions?
• Query types: Short factual questions? Multi-step reasoning queries? Semantic search? Code search?
• Latency requirements: Real-time chat ( "Use the RAG Workflow Planner skill. I'm building a chatbot that answers questions about our company's internal policy documents. We have about 200 PDFs, mostly 5–20 pages each. Queries will be short questions like 'What is the vacation policy?' or 'How do I expense international travel?' Response time under 3 seconds. We're on AWS, prefer managed services where possible."

Output from Agent: > ## RAG Architecture for Internal Policy Q&A Chatbot > > ### Requirements Summary > - 200 PDFs, 5–20 pages each (~500 total pages) > - Short factual English queries > - - AWS, prefer managed services > - Static corpus with occasional updates > > ### Architecture Diagram > > `` > [PDF Files] > │ > ▼ > [PDF Parser + Text Cleaner] > │ > ▼ > [Semantic Chunker] ──── ~500 token chunks, paragraph boundaries > │ > ▼ > [OpenAI text-embedding-3-small] > │ > ▼ > [Pinecone Index] ◄──────────────────────────────────┐ > │ > [User Query] ──► [Embed Query] ──► [Top-20 Retrieve] ──► [Cohere Rerank top-5] ──► [GPT-4o] ──► [Answer + Sources] > ` > > ### Component Breakdown > > | Component | Recommendation | Justification | > |-----------|---------------|---------------| > | Chunking | Semantic (paragraph boundaries, ~500 tokens, 50 token overlap) | Policy docs have clear paragraph structure; semantic chunks preserve policy context | > | Embedding | OpenAI text-embedding-3-small | Best accuracy/cost for English, managed, no infra | > | Vector Store | Pinecone Serverless | AWS-native, managed, scales to this corpus size cheaply | > | Retrieval | Dense (cosine similarity), top-20 | Semantic queries match well; 200 docs don't need hybrid | > | Reranker | Cohere Rerank v3, top-5 | Improves precision significantly for policy Q&A; | LLM | GPT-4o or Claude 3 Haiku | Haiku for speed, GPT-4o for complex multi-policy questions | > > ### Ingestion Pipeline > 1. Load PDF → extract text with pdfplumber or AWS Textract (for scanned PDFs) > 2. Clean: strip page numbers, headers, footers > 3. Split on paragraph boundaries; merge short paragraphs until ~500 tokens > 4. Embed each chunk with text-embedding-3-small > 5. Store in Pinecone with metadata: { docname, pagerange, lastupdated } > > ### Query Pipeline > 1. Receive user query > 2. Embed with text-embedding-3-small > 3. Retrieve top-20 chunks from Pinecone > 4. Rerank with Cohere Rerank → keep top-5 > 5. Build prompt: system instructions + top-5 chunks as context + user question > 6. Call LLM; instruct it to cite the source document name > 7. Return answer + source list > > ### Risks and Mitigations > - **Stale documents:** Re-embed changed PDFs on upload; store lastupdated` in metadata and filter stale chunks if needed > - Hallucination: System prompt: "Answer only from the provided context. If the answer is not in the documents, say so." > - Low retrieval recall: If reranker still misses answers, add query expansion: generate 3 query variants with the LLM before retrieval

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Notysoty
• Source: Notysoty/openagentskills
• License: MIT

Install and usage instructions live in the source repository linked above.