General Graph Extraction

General Graph Extraction Skill

Agent Runtime Contract

Use this skill when the raw input file is in to_be_extracted/ and the content is not primarily financial or academic literature.

Execution contract:

• Call get_server_status() before starting extraction.
• If overall is degraded, explain what is degraded, what still works, and ask whether to proceed or fix first.
• Read one raw source file from to_be_extracted/ at a time unless the user explicitly asks for a combined extraction.
• For the best graph quality, prefer .md and .txt source files. If the source is PDF or heavily formatted export text, assume there is no built-in repo parser or OCR layer and be conservative about layout noise.
• Write exactly one extraction JSON for that source file to extractions/{source_filename}_extracted.json.
• Validate entity, relationship, and chunk integrity before ingestion.
• If the extraction has duplicates, orphaned relationships, or consistency issues, run reconciliation before ingestion.
• After a clean extraction is written, call ingest_from_file.

What This Skill Does

This skill guides an agent to read a document or codebase and produce a structured JSON extraction — entities and relationships — that can be ingested into a knowledge graph via the ingest_from_file MCP tool.

Unlike financial extraction (which enforces strict numerical precision), general extraction prioritizes semantic richness and connection density — the goal is to enable natural-language queries like:

• "What does the AuthService depend on?"
• "Which modules are owned by the payments team?"
• "What broke when we changed the database schema?"
• "Who are the experts on the recommendation engine?"

The goal is not to graph every noun. The goal is to build a graph with high-signal, reusable structure:

• who owns what
• what depends on what
• what decisions were made
• what actions, risks, blockers, and systems matter
• what evidence supports those links

---

General Reading Strategy

Read the source in layers instead of linearly dumping entities:

Pass 1 — Identify document subtype and signal

Classify the source before extracting:

• codebase or technical docs
• internal wiki or knowledge base
• meeting notes or ticket
• policy or process document
• product spec or decision record

During this pass, decide:

• what the main units of knowledge are
• what should count as entities versus plain context
• which sections are likely high signal
• whether the source is clean enough for dense extraction

Pass 2 — Extract structure first

Prioritize:

• named systems, modules, APIs, tools, teams, people, processes, decisions, projects
• explicit ownership
• explicit dependencies
• procedural order
• blockers, action items, and operational constraints

Pass 3 — Add only useful semantic detail

Add broader concepts only when they help downstream queries:

• architecture patterns
• operational policies
• domain concepts
• known issues

If the text is noisy, repetitive, or weakly structured, reduce extraction density instead of inflating the graph.

---

Extraction Output Format (Repo-Compatible)

Write to extractions/{source_filename}_extracted.json.

This repo requires document_id, entities, relationships, and chunks.

{
  "document_id": "backend_readme",
  "file_path": "README.md",
  "timestamp": 1727740800,
  "entities": [...],
  "relationships": [...],
  "chunks": [...]
}

Chunks (Required)

Chunks are the evidence base. Every entity and relationship must point to a chunk via source_id.

{
  "chunk_id": "chunk_001",
  "content": "The OrderService is owned by the Payments team and calls InventoryService.",
  "chunk_order_index": 1,
  "file_path": "README.md"
}

**Chunk size limits (required):**
- Target 400-800 characters per chunk (or ~200-300 tokens if you have a tokenizer).
- Hard cap 1,000 characters per chunk. If a paragraph/table is longer, split it.

---

Domain Subtypes and Entity Profiles

Choose the right entity profile based on what you're processing:

1. Codebase / Technical Docs

| Entity Type | Examples | |-------------|---------| | MODULE | auth_service, payment_processor | | FUNCTION | validate_token(), create_order() | | CLASS | UserRepository, InvoiceBuilder | | API_ENDPOINT | POST /api/v2/orders | | DATABASE_TABLE | users, transactions | | CONFIG_KEY | MAX_RETRY_COUNT, DB_URL | | DEPENDENCY | numpy, FastAPI, Redis | | TEAM | Platform Team, Payments Squad | | CONCEPT | idempotency, circuit breaker |

Relevant relationship keywords: IMPORTS, CALLS, OWNS, DEPENDS_ON, DOCUMENTS, IMPLEMENTS, EXTENDS, STORES_IN, EXPOSES

2. Internal Wiki / Knowledge Base

| Entity Type | Examples | |-------------|---------| | PERSON | Sanjay Elango (author/owner) | | TEAM | Backend Infrastructure | | PROCESS | Deployment Pipeline, Oncall Rotation | | TOOL | Datadog, PagerDuty, Terraform | | CONCEPT | SLA definition, retry policy | | DOCUMENT | Runbook: DB failover | | DECISION | ADR-042: Use Postgres over MySQL |

Relevant relationship keywords: OWNS, WRITTEN_BY, REFERENCES, SUPERSEDES, IMPLEMENTS, USES, DESCRIBES, DECIDES

3. Meeting Notes / Tickets

| Entity Type | Examples | |-------------|---------| | PERSON | Named attendees/assignees | | ACTION_ITEM | "Fix token expiry bug by Friday" | | DECISION | "We'll use Redis for session storage" | | BLOCKER | "Waiting on security review" | | PROJECT | Project Phoenix | | DATE | 2024-10-15 (target date) |

Relevant relationship keywords: ASSIGNED_TO, BLOCKS, RELATES_TO, DECIDED_IN, DUE_BY

---

Universal Entity Schema (Required Fields)

{
  "entity_name": "auth_service",
  "entity_type": "MODULE",
  "description": "Authentication service that issues and validates JWTs; owner=platform_team; path=services/auth/.",
  "source_id": "chunk_002",
  "file_path": "services/auth/README.md"
}

Rules:

• entity_name: unique, snake_case, no spaces
• entity_type: pick from the relevant profile list
• description: include concrete facts and properties from the text (owner, path, constraints)
• source_id: must match a real chunk_id in chunks
• prefer one strong entity per real concept, not many aliases for the same thing

---

Universal Relationship Schema (Required Fields)

{
  "src_id": "order_service",
  "tgt_id": "payment_processor",
  "description": "OrderService calls PaymentProcessor to charge the customer during checkout.",
  "keywords": "CALLS,context=checkout_flow",
  "source_id": "chunk_001",
  "weight": 1.0,
  "file_path": "README.md"
}

Rules:

• src_id and tgt_id must be valid entity_name values
• keywords should include the relationship type (e.g., CALLS, OWNS, DEPENDS_ON)
• Add context=... or source_section=... in keywords when useful
• Directed relationships: source → target; for bidirectional add both directions
• only create a relationship when the source text gives direct support for it or the implication is extremely clear from local context

---

What To Ignore

Do not extract:

• repeated headers, navigation text, sidebars, export boilerplate, template labels
• generic filler like "this document describes" unless it carries a real relationship
• every file path, import, or config key in bulk when they add no query value
• every attendee name in notes if they are not tied to a decision, action, or ownership signal
• duplicate concepts restated in multiple sections

Prefer signal over completeness. A smaller, cleaner graph is better than a bloated one.

---

Extraction Quality Principles

For Codebases

• Trace dependencies: what imports what, what calls what
• Map ownership: which team/person owns which module
• Surface configuration: key env vars and their consuming modules
• Capture architectural patterns mentioned (MVC, event-driven, etc.)
• Note TODOs/FIXMEs as KNOWN_ISSUE entities if they're significant
• Distinguish between implementation details and stable concepts; prefer stable concepts unless the user clearly wants code-level granularity

For Documentation

• Every named concept becomes an entity
• Every "see also", "refer to", "documented in" becomes a REFERENCES relationship
• Every "owned by", "maintained by" becomes an OWNS relationship
• Procedural steps → sequence of PROCESS entities linked by PRECEDES
• Collapse repeated explanations into one canonical entity with the clearest supporting chunk

For Notes/Tickets

• Every named person is an entity
• Every task or action item is an entity
• Capture blockers as explicit BLOCKER entities with BLOCKS relationships
• Favor decisions, owners, due dates, and blockers over conversational filler

---

Extraction Density vs. Precision Trade-off

| Dimension | Financial Extraction | General Extraction | |-----------|---------------------|--------------------| | Precision | Exact (numbers must match) | Semantic (meaning over literal match) | | Coverage | Deep in financial semantics | Broad across any domain | | Ambiguity | Resolve strictly (no inference) | Light inference OK if clearly implied | | Missing values | Leave blank, don't guess | Infer from context if confident |

In general extraction, if a README says "the auth module handles login and token refresh", it's reasonable to extract a CONCEPT entity token_refresh and link it to the auth module via IMPLEMENTS even if not explicitly stated that way.

But do not stretch this into speculation. If the text is ambiguous, prefer fewer entities and relationships.

---

Better Chunking For General Sources

Use chunk boundaries that preserve meaning:

• one architectural paragraph
• one decision paragraph
• one action-item block
• one API section
• one process step group

Avoid chunks that mix unrelated topics just to hit a length target. Semantic coherence matters more than uniform size.

---

Better Graph Outcomes

Optimize extraction for the questions users actually ask later:

• ownership: who owns this system, process, or document
• dependency: what calls, uses, imports, or relies on what
• decision history: what changed, why, and what it replaced
• operations: what blocks what, what must happen first, what is risky
• expertise: who is associated with which subsystem or domain

If an extracted entity or relationship is unlikely to help answer one of those question types, think twice before adding it.

---

Step-by-Step Agent Workflow

1. CALL get_server_status() and surface degraded modes before continuing
2. READ the full document (or repo files in scope)
3. IDENTIFY domain subtype (codebase, wiki, notes, etc.)
4. CHUNK the text into evidence blocks (sections, paragraphs, or tables)
   - Assign chunk_id like chunk_001, chunk_002...
5. SCAN for all named entities — people, modules, tools, concepts, processes
6. EXTRACT entities with required fields and chunk-based source_id
7. EXTRACT relationships by asking: "how does X relate to Y in this text?"
8. VALIDATE:
   - No entity_name duplicates
   - All relationships reference existing entity_name values
   - Every entity/relationship source_id matches a chunk_id
   - Output path matches the current source file name
9. WRITE to extractions/{filename}_extracted.json
10. CALL ingest_from_file MCP tool

---

Example: Codebase README Extraction

Input text: > "The OrderService module is owned by the Payments team. It calls InventoryService to check stock before calling PaymentProcessor to charge the customer. It stores order records in the orders table of the PostgreSQL database."

Output:

{
  "document_id": "backend_readme",
  "file_path": "README.md",
  "timestamp": 1727740800,
  "chunks": [
    {
      "chunk_id": "chunk_001",
      "content": "The OrderService module is owned by the Payments team. It calls InventoryService to check stock before calling PaymentProcessor to charge the customer. It stores order records in the orders table of the PostgreSQL database.",
      "chunk_order_index": 1,
      "file_path": "README.md"
    }
  ],
  "entities": [
    {
      "entity_name": "order_service",
      "entity_type": "MODULE",
      "description": "OrderService module owned by Payments team.",
      "source_id": "chunk_001",
      "file_path": "README.md"
    },
    {
      "entity_name": "inventory_service",
      "entity_type": "MODULE",
      "description": "InventoryService called to check stock before charging.",
      "source_id": "chunk_001",
      "file_path": "README.md"
    },
    {
      "entity_name": "payment_processor",
      "entity_type": "MODULE",
      "description": "PaymentProcessor used to charge the customer.",
      "source_id": "chunk_001",
      "file_path": "README.md"
    },
    {
      "entity_name": "payments_team",
      "entity_type": "TEAM",
      "description": "Payments team owns OrderService.",
      "source_id": "chunk_001",
      "file_path": "README.md"
    },
    {
      "entity_name": "orders_table",
      "entity_type": "DATABASE_TABLE",
      "description": "orders table in PostgreSQL storing order records.",
      "source_id": "chunk_001",
      "file_path": "README.md"
    }
  ],
  "relationships": [
    {
      "src_id": "payments_team",
      "tgt_id": "order_service",
      "description": "Payments team owns OrderService.",
      "keywords": "OWNS",
      "source_id": "chunk_001",
      "weight": 1.0,
      "file_path": "README.md"
    },
    {
      "src_id": "order_service",
      "tgt_id": "inventory_service",
      "description": "OrderService calls InventoryService to check stock.",
      "keywords": "CALLS,context=stock_check",
      "source_id": "chunk_001",
      "weight": 1.0,
      "file_path": "README.md"
    },
    {
      "src_id": "order_service",
      "tgt_id": "payment_processor",
      "description": "OrderService calls PaymentProcessor to charge the customer.",
      "keywords": "CALLS,context=checkout_flow",
      "source_id": "chunk_001",
      "weight": 1.0,
      "file_path": "README.md"
    },
    {
      "src_id": "order_service",
      "tgt_id": "orders_table",
      "description": "OrderService stores order records in orders table.",
      "keywords": "STORES_IN,db=PostgreSQL",
      "source_id": "chunk_001",
      "weight": 1.0,
      "file_path": "README.md"
    }
  ]
}

---

After Extraction

• Run ingest_from_file("extractions/{filename}_extracted.json") via MCP
• If ingestion fails, use reingest_from_file(...) to replay without re-extracting
• Sample queries after ingestion:
• "What does OrderService depend on?"
• "Which tables does the Payments team write to?"
• "Who owns InventoryService?"

---

Large Document Handling

When to use subagents

Never for extraction. Always extract the full document yourself. Your entity registry prevents name drift.

Spawn reconciliation subagents only when:

• Extraction produces more than 150 entities
• Extraction produces more than 300 relationships
• You want a quality check before ingestion

How to spawn reconciliation subagents

After writing your extraction file:

Step 1: Spawn 3 reconciliation subagents in parallel. Give each subagent the extraction JSON.

Subagent 1 gets: entities list only Task: find entity name variants and duplicates Output: extractions/{filename}patchentities.json

Subagent 2 gets: relationships list only Task: find duplicate relationships and conflicts Output: extractions/{filename}patchrelationships.json

Subagent 3 gets: entities + relationships Task: find broken references (srcid or tgtid not in entities list) Output: extractions/{filename}patchorphans.json

Step 2: Wait for all 3 subagents to finish.

Step 3: Call ingestwithreconciliation([ "extractions/{filename}_extr

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Preciso-GR
• Source: Preciso-GR/preciso-graphrag
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.