Skill Seekers

> Want to contribute? The website and configs repos are great starting points for new contributors!

🧠 The Data Layer for AI Systems

Skill Seekers is the universal preprocessing layer that sits between raw documentation and every AI system that consumes it. Whether you are building Claude skills, a LangChain RAG pipeline, or a Cursor .cursorrules file — the data preparation is identical. You do it once, and export to all targets.

# One command → structured knowledge asset
skill-seekers create https://docs.react.dev/
# or: skill-seekers create facebook/react
# or: skill-seekers create ./my-project

# Export to any AI system
skill-seekers package output/react --target claude      # → Claude AI Skill (ZIP)
skill-seekers package output/react --target langchain   # → LangChain Documents
skill-seekers package output/react --target llama-index # → LlamaIndex TextNodes
skill-seekers package output/react --target cursor      # → .cursorrules
skill-seekers package output/react --target ibm-bob     # → IBM Bob skill directory

What gets built

| Output | Target | What it powers | |--------|--------|---------------| | Claude Skill (ZIP + YAML) | --target claude | Claude Code, Claude API | | Gemini Skill (tar.gz) | --target gemini | Google Gemini | | OpenAI / Custom GPT (ZIP) | --target openai | GPT-4o, custom assistants | | LangChain Documents | --target langchain | QA chains, agents, retrievers | | LlamaIndex TextNodes | --target llama-index | Query engines, chat engines | | Haystack Documents | --target haystack | Enterprise RAG pipelines | | Pinecone-ready (Markdown) | --target markdown | Vector upsert | | ChromaDB / FAISS / Qdrant | --target chroma/faiss/qdrant | Local vector DBs | | IBM Bob Skill (directory) | --target ibm-bob | IBM Bob project/global skills | | Cursor .cursorrules | --target markdown → copy SKILL.md | Cursor IDE .cursorrules | | Windsurf / Cline / Continue | --target claude → copy | VS Code, IntelliJ, Vim |

Why it matters

• ⚡ 99% faster — Days of manual data prep → 15–45 minutes
• 🎯 AI Skill quality — 500+ line SKILL.md files with examples, patterns, and guides
• 📊 RAG-ready chunks — Smart chunking preserves code blocks and maintains context
• 🎬 Videos — Extract code, transcripts, and structured knowledge from YouTube and local videos
• 🔄 Multi-source — Combine 18 source types (docs, GitHub, PDFs, videos, notebooks, wikis, and more) into one knowledge asset
• 🌐 One prep, every target — Export the same asset to 21 platforms without re-scraping
• ✅ Battle-tested — 3,700+ tests, 24+ framework presets, production-ready

🚀 Quick Start (3 Commands)

# 1. Install
pip install skill-seekers

# 2. Create skill from any source
skill-seekers create https://docs.django.com/

# 3. Package for your AI platform
skill-seekers package output/django --target claude

That's it! You now have output/django-claude.zip ready to use.

# Use a different AI agent for enhancement (default: claude)
skill-seekers create https://docs.django.com/ --agent kimi
skill-seekers create https://docs.django.com/ --agent codex
skill-seekers create https://docs.django.com/ --agent-cmd "my-custom-agent run"

🛰️ AI-driven project scan (new)

Point scan at any project and an AI agent reads its manifests, README, Dockerfile/CI and sampled source imports — then emits one config per detected framework plus a -codebase.json for your own code. Pins the detected version so re-running reports bumps:

skill-seekers scan ./my-react-app --out ./configs/scanned/
# → react.json, vite.json, tailwind.json, jest.json, my-react-app-codebase.json

# Then build any of them
skill-seekers create ./configs/scanned/react.json

If a detection has no existing preset, the AI generates a fresh config; on exit you can optionally publish it back to the community registry.

Other Sources (18 Supported)

# GitHub repository
skill-seekers create facebook/react

# Local project
skill-seekers create ./my-project

# PDF document
skill-seekers create manual.pdf

# Word document
skill-seekers create report.docx

# EPUB e-book
skill-seekers create book.epub

# Jupyter Notebook
skill-seekers create notebook.ipynb

# OpenAPI spec
skill-seekers create openapi.yaml

# PowerPoint presentation
skill-seekers create presentation.pptx

# AsciiDoc document
skill-seekers create guide.adoc

# Local HTML file (auto-detected by extension)
skill-seekers create page.html

# Whole directory of HTML files (auto-detected for HTML-dominant dirs)
skill-seekers create ./mirror_output/site/

# Force HTML mode on a mixed/code-heavy directory
skill-seekers create ./repo/ --html-path ./repo/docs/build/html/

# RSS/Atom feed
skill-seekers create feed.rss

# Man page
skill-seekers create curl.1

# Video (YouTube, Vimeo, or local file — requires skill-seekers[video])
skill-seekers create --video-url https://www.youtube.com/watch?v=... --name mytutorial
# First time? Auto-install GPU-aware visual deps:
skill-seekers create --setup

# Confluence wiki
skill-seekers create --space-key TEAM --name wiki

# Notion pages
skill-seekers create --database-id ... --name docs

# Slack/Discord chat export
skill-seekers create --chat-export-path ./slack-export --name team-chat

Export Everywhere

# Package for multiple platforms
for platform in claude gemini openai langchain; do
  skill-seekers package output/django --target $platform
done

What is Skill Seekers?

Skill Seekers is the data layer for AI systems. It transforms 18 source types—documentation websites, GitHub repositories, PDFs, videos, Jupyter Notebooks, Word/EPUB/AsciiDoc documents, OpenAPI specs, PowerPoint presentations, RSS feeds, man pages, Confluence wikis, Notion pages, Slack/Discord exports, and more—into structured knowledge assets for every AI target:

| Use Case | What you get | Examples | |----------|-------------|---------| | AI Skills | Comprehensive SKILL.md + references | Claude Code, Gemini, GPT | | RAG Pipelines | Chunked documents with rich metadata | LangChain, LlamaIndex, Haystack | | Vector Databases | Pre-formatted data ready for upsert | Pinecone, Chroma, Weaviate, FAISS | | AI Coding Assistants | Context files your IDE AI reads automatically | Cursor, Windsurf, Cline, Continue.dev |

📚 Documentation

| I want to... | Read this | |--------------|-----------| | Get started quickly | [Quick Start](docs/getting-started/02-quick-start.md) - 3 commands to first skill | | Understand concepts | [Core Concepts](docs/user-guide/01-core-concepts.md) - How it works | | Scrape sources | [Scraping Guide](docs/user-guide/02-scraping.md) - All source types | | Enhance skills | [Enhancement Guide](docs/user-guide/03-enhancement.md) - AI enhancement | | Export skills | [Packaging Guide](docs/user-guide/04-packaging.md) - Platform export | | Look up commands | [CLI Reference](docs/reference/CLIREFERENCE.md) - All 20 commands | | Configure | [Config Format](docs/reference/CONFIGFORMAT.md) - JSON specification | | Fix issues | [Troubleshooting](docs/user-guide/06-troubleshooting.md) - Common problems |

Complete documentation: [docs/README.md](docs/README.md)

Instead of spending days on manual preprocessing, Skill Seekers:

1. Ingests — docs, GitHub repos, local codebases, PDFs, videos, notebooks, wikis, and 10+ more source types
2. Analyzes — deep AST parsing, pattern detection, API extraction
3. Structures — categorized reference files with metadata
4. Enhances — AI-powered SKILL.md generation (Claude, Gemini, or local)
5. Exports — 16 platform-specific formats from one asset

Why Use This?

For AI Skill Builders (Claude, Gemini, OpenAI)

• 🎯 Production-grade Skills — 500+ line SKILL.md files with code examples, patterns, and guides
• 🔄 Enhancement Workflows — Apply security-focus, architecture-comprehensive, or custom YAML presets
• 🎮 Any Domain — Game engines (Godot, Unity), frameworks (React, Django), internal tools
• 🔧 Teams — Combine internal docs + code into a single source of truth
• 📚 Quality — AI-enhanced with examples, quick reference, and navigation guidance

For RAG Builders & AI Engineers

• 🤖 RAG-ready data — Pre-chunked LangChain Documents, LlamaIndex TextNodes, Haystack Documents
• 🚀 99% faster — Days of preprocessing → 15–45 minutes
• 📊 Smart metadata — Categories, sources, types → better retrieval accuracy
• 🔄 Multi-source — Combine docs + GitHub + PDFs + videos in one pipeline
• 🌐 Platform-agnostic — Export to any vector DB or framework without re-scraping

For AI Coding Assistant Users

• 💻 Cursor / Windsurf / Cline — Generate .cursorrules / .windsurfrules / .clinerules automatically
• 🎯 Persistent context — AI "knows" your frameworks without repeated prompting
• 📚 Always current — Update context in minutes when docs change

Key Features

🌐 Documentation Scraping

• ✅ Smart SPA Discovery - Three-layer discovery for JavaScript SPA sites (sitemap.xml → llms.txt → headless browser rendering)
• ✅ llms.txt Support - Automatically detects and uses LLM-ready documentation files (10x faster)
• ✅ Universal Scraper - Works with ANY documentation website
• ✅ Smart Categorization - Automatically organizes content by topic
• ✅ Code Language Detection - Recognizes Python, JavaScript, C++, GDScript, etc.
• ✅ 24+ Ready-to-Use Presets - Godot, React, Vue, Django, FastAPI, and more

📄 PDF Support

• ✅ Basic PDF Extraction - Extract text, code, and images from PDF files
• ✅ OCR for Scanned PDFs - Extract text from scanned documents
• ✅ Password-Protected PDFs - Handle encrypted PDFs
• ✅ Table Extraction - Extract complex tables from PDFs
• ✅ Parallel Processing - 3x faster for large PDFs
• ✅ Intelligent Caching - 50% faster on re-runs

🎬 Video Extraction

• ✅ YouTube & Local Videos - Extract transcripts, on-screen code, and structured knowledge from videos
• ✅ Visual Frame Analysis - OCR extraction from code editors, terminals, slides, and diagrams
• ✅ GPU Auto-Detection - Automatically installs correct PyTorch build (CUDA/ROCm/MPS/CPU)
• ✅ AI Enhancement - Two-pass: clean OCR artifacts + generate polished SKILL.md
• ✅ Time Clipping - Extract specific sections with --start-time and --end-time
• ✅ Playlist Support - Batch process all videos in a YouTube playlist
• ✅ Vision API Fallback - Use Claude Vision for low-confidence OCR frames

🐙 GitHub Repository Analysis

• ✅ Deep Code Analysis - AST parsing for Python, JavaScript, TypeScript, Java, C++, Go
• ✅ API Extraction - Functions, classes, methods with parameters and types
• ✅ Repository Metadata - README, file tree, language breakdown, stars/forks
• ✅ GitHub Issues & PRs - Fetch open/closed issues with labels and milestones
• ✅ CHANGELOG & Releases - Automatically extract version history
• ✅ Conflict Detection - Compare documented APIs vs actual code implementation
• ✅ MCP Integration - Natural language: "Scrape GitHub repo facebook/react"

🔄 Unified Multi-Source Scraping

• ✅ Combine Multiple Sources - Mix documentation + GitHub + PDF in one skill
• ✅ Conflict Detection - Automatically finds discrepancies between docs and code
• ✅ Intelligent Merging - Rule-based or AI-powered conflict resolution
• ✅ Transparent Reporting - Side-by-side comparison with ⚠️ warnings
• ✅ Documentation Gap Analysis - Identifies outdated docs and undocumented features
• ✅ Single Source of Truth - One skill showing both intent (docs) and reality (code)
• ✅ Backward Compatible - Legacy single-source configs still work

🤖 Multi-LLM Platform Support

• ✅ 12 LLM Platforms - Claude AI, Google Gemini, OpenAI ChatGPT, MiniMax AI, Generic Markdown, OpenCode, Kimi (Moonshot AI), DeepSeek AI, Qwen (Alibaba), OpenRouter, Together AI, Fireworks AI
• ✅ Universal Scraping - Same documentation works for all platforms
• ✅ Platform-Specific Packaging - Optimized formats for each LLM
• ✅ One-Command Export - --target flag selects platform
• ✅ Optional Dependencies - Install only what you need
• ✅ 100% Backward Compatible - Existing Claude workflows unchanged

| Platform | Format | Upload | Enhancement | API Key | Custom Endpoint | |----------|--------|--------|-------------|---------|-----------------| | Claude AI | ZIP + YAML | ✅ Auto | ✅ Yes | ANTHROPICAPIKEY | ANTHROPICBASEURL | | Google Gemini | tar.gz | ✅ Auto | ✅ Yes | GOOGLEAPIKEY | - | | OpenAI ChatGPT | ZIP + Vector Store | ✅ Auto | ✅ Yes | OPENAIAPIKEY | - | | MiniMax AI | ZIP + Knowledge Files | ✅ Auto | ✅ Yes | MINIMAXAPIKEY | - | | Generic Markdown | ZIP | ❌ Manual | ❌ No | - | - |

# Claude (default - no changes needed!)
skill-seekers package output/react/
skill-seekers upload react.zip

# Google Gemini
pip install skill-seekers[gemini]
skill-seekers package output/react/ --target gemini
skill-seekers upload react-gemini.tar.gz --target gemini

# OpenAI ChatGPT
pip install skill-seekers[openai]
skill-seekers package output/react/ --target openai
skill-seekers upload react-openai.zip --target openai

# MiniMax AI
pip install skill-seekers[minimax]
skill-seekers package output/react/ --target m

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [yusufkaraaslan](https://github.com/yusufkaraaslan)
- **Source:** [yusufkaraaslan/Skill_Seekers](https://github.com/yusufkaraaslan/Skill_Seekers)
- **License:** MIT
- **Homepage:** https://skillseekersweb.com/

Install and usage instructions live in the source repository linked above.