Graphify

/graphify

Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md.

Usage

/graphify                                             # full pipeline on current directory → Obsidian vault
/graphify                                       # full pipeline on specific path
/graphify  --mode deep                          # thorough extraction, richer INFERRED edges
/graphify  --update                             # incremental - re-extract only new/changed files
/graphify  --directed                            # build directed graph (preserves edge direction: source→target)
/graphify  --whisper-model medium                # use a larger Whisper model for better transcription accuracy
/graphify  --cluster-only                       # rerun clustering on existing graph
/graphify  --no-viz                             # skip visualization, just report + JSON
/graphify  --html                               # (HTML is generated by default - this flag is a no-op)
/graphify  --svg                                # also export graph.svg (embeds in Notion, GitHub)
/graphify  --graphml                            # export graph.graphml (Gephi, yEd)
/graphify  --neo4j                              # generate graphify-out/cypher.txt for Neo4j
/graphify  --neo4j-push bolt://localhost:7687   # push directly to Neo4j
/graphify  --mcp                                # start MCP stdio server for agent access
/graphify  --watch                              # watch folder, auto-rebuild on code changes (no LLM needed)
/graphify  --wiki                               # build agent-crawlable wiki (index.md + one article per community)
/graphify  --obsidian --obsidian-dir ~/vaults/my-project  # write vault to custom path (e.g. existing vault)
/graphify add                                    # fetch URL, save to ./raw, update graph
/graphify add  --author "Name"                   # tag who wrote it
/graphify add  --contributor "Name"              # tag who added it to the corpus
/graphify query ""                          # BFS traversal - broad context
/graphify query "" --dfs                    # DFS - trace a specific path
/graphify query "" --budget 1500            # cap answer at N tokens
/graphify path "AuthModule" "Database"                # shortest path between two concepts
/graphify explain "SwinTransformer"                   # plain-language explanation of a node

What graphify is for

graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected.

Three things it does that Claude alone cannot:

1. Persistent graph - relationships are stored in graphify-out/graph.json and survive across sessions. Ask questions weeks later without re-reading everything.
2. Honest audit trail - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented.
3. Cross-document surprise - community detection finds connections between concepts in different files that you would never think to ask about directly.

Use it for:

• A codebase you're new to (understand architecture before touching anything)
• A reading list (papers + tweets + notes → one navigable graph)
• A research corpus (citation graph + concept graph in one)
• Your personal /raw folder (drop everything in, let it grow, query it)

What You Must Do When Invoked

CRITICAL - SUBCOMMAND DISPATCH: Before doing anything else, parse the user's command to identify the subcommand. Follow this table:

| User's command | Action | |---|---| | /graphify query "..." | Jump to "For /graphify query" section below. Do NOT run the full pipeline. | | /graphify path "A" "B" | Jump to "For /graphify path" section below. Do NOT run the full pipeline. | | /graphify explain "..." | Jump to "For /graphify explain" section below. Do NOT run the full pipeline. | | /graphify add | Jump to "For /graphify add" section below. Do NOT run the full pipeline. | | /graphify --update | Jump to "For --update" section below. | | /graphify --cluster-only | Jump to "For --cluster-only" section below. | | /graphify --watch | Jump to "For --watch" section below. | | /graphify or /graphify (with optional flags) | Run the full pipeline (Steps 0–9) below. |

If the subcommand is query, path, explain, or add, you MUST skip the full pipeline entirely and go directly to the corresponding section. These subcommands operate on an EXISTING graph — they do NOT build a new one.

If no path was given, use . (current directory). Do not ask the user for a path.

IMPORTANT: Always start by setting the working directory to ~/Documents/wiki to ensure graphify-out is created in the correct location.

Follow these steps in order. Do not skip steps.

Step 0 - Set Working Directory

cd ~/Documents/wiki || { echo "Error: ~/Documents/wiki directory not found. Creating it..."; mkdir -p ~/Documents/wiki; cd ~/Documents/wiki; }

Step 1 - Ensure graphify is installed

cd ~/Documents/wiki
# Detect the correct Python interpreter (handles pipx, venv, system installs)
GRAPHIFY_BIN=$(which graphify 2>/dev/null)
if [ -n "$GRAPHIFY_BIN" ]; then
    PYTHON=$(head -1 "$GRAPHIFY_BIN" | tr -d '#!')
    case "$PYTHON" in
        *[!a-zA-Z0-9/_.-]*) PYTHON="python3" ;;
    esac
else
    PYTHON="python3"
fi
"$PYTHON" -c "import graphify" 2>/dev/null || "$PYTHON" -m pip install graphifyy -q 2>/dev/null || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3
# Write interpreter path for all subsequent steps (persists across invocations)
mkdir -p graphify-out
"$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w').write(sys.executable)"

If the import succeeds, print nothing and move straight to Step 2.

In every subsequent bash block, start with cd ~/Documents/wiki and then replace python3 with $(cat graphify-out/.graphify_python) to use the correct interpreter.

Step 2 - Detect files

cd ~/Documents/wiki
$(cat graphify-out/.graphify_python) -c "
import json
from graphify.detect import detect
from pathlib import Path
result = detect(Path('INPUT_PATH'))
print(json.dumps(result))
" > graphify-out/.graphify_detect.json

Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead:

Corpus: X files · ~Y words
  code:     N files (.py .ts .go ...)
  docs:     N files (.md .txt ...)
  papers:   N files (.pdf ...)
  images:   N files
  video:    N files (.mp4 .mp3 ...)

Omit any category with 0 files from the summary.

Then act on it:

• If total_files is 0: stop with "No supported files found in [path]."
• If skipped_sensitive is non-empty: mention file count skipped, not the file names.
• If total_words > 2,000,000 OR total_files > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding.
• Otherwise: proceed directly to Step 2.5 if video files were detected, or Step 3 if not.

Step 2.5 - Transcribe video / audio files (only if video files detected)

Skip this step entirely if detect returned zero video files.

Video and audio files cannot be read directly. Transcribe them to text first, then treat the transcripts as doc files in Step 3.

Strategy: Read the god nodes from graphify-out/.graphify_detect.json (or the analysis file if it exists from a previous run). You are already a language model — write a one-sentence domain hint yourself from those labels. Then pass it to Whisper as the initial prompt. No separate API call needed.

However, if the corpus has only video files and no other docs/code, use the generic fallback prompt: "Use proper punctuation and paragraph breaks."

Step 1 - Write the Whisper prompt yourself.

Read the top god node labels from detect output or analysis, then compose a short domain hint sentence, for example:

• Labels: transformer, attention, encoder, decoder → "Machine learning research on transformer architectures and attention mechanisms. Use proper punctuation and paragraph breaks."
• Labels: kubernetes, deployment, pod, helm → "DevOps discussion about Kubernetes deployments and Helm charts. Use proper punctuation and paragraph breaks."

Set it as WHISPER_PROMPT to use in the next command.

Step 2 - Transcribe:

GRAPHIFY_WHISPER_MODEL=base  # or whatever --whisper-model the user passed
$(cat graphify-out/.graphify_python) -c "
import json, os
from pathlib import Path
from graphify.transcribe import transcribe_all

detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
video_files = detect.get('files', {}).get('video', [])
prompt = os.environ.get('GRAPHIFY_WHISPER_PROMPT', 'Use proper punctuation and paragraph breaks.')

transcript_paths = transcribe_all(video_files, initial_prompt=prompt)
print(json.dumps(transcript_paths))
" > graphify-out/.graphify_transcripts.json

After transcription:

• Read the transcript paths from graphify-out/.graphify_transcripts.json
• Add them to the docs list before dispatching semantic subagents in Step 3B
• Print how many transcripts were created: Transcribed N video file(s) -> treating as docs
• If transcription fails for a file, print a warning and continue with the rest

Whisper model: Default is base. If the user passed --whisper-model , set GRAPHIFY_WHISPER_MODEL= in the environment before running the command above.

Step 3 - Extract entities and relationships

Before starting: note whether --mode deep was given. You must pass DEEP_MODE=true to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it.

This step has two parts: structural extraction (deterministic, free) and semantic extraction (Claude, costs tokens).

Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.

Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers.

Part A - Structural extraction for code files

For any code files detected, run AST extraction in parallel with Part B subagents:

cd ~/Documents/wiki
$(cat graphify-out/.graphify_python) -c "
import sys, json
from graphify.extract import collect_files, extract
from pathlib import Path
import json

code_files = []
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
for f in detect.get('files', {}).get('code', []):
    code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)])

if code_files:
    result = extract(code_files)
    Path('graphify-out/.graphify_ast.json').write_text(json.dumps(result, indent=2))
    print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges')
else:
    Path('graphify-out/.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0}))
    print('No code files - skipping AST extraction')
"

Part B - Semantic extraction (parallel subagents)

Fast path: If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do.

MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.

Before dispatching subagents, print a timing estimate:

• Load total_words and file counts from graphify-out/.graphify_detect.json
• Estimate agents needed: ceil(uncached_non_code_files / 22) (chunk size is 20-25)
• Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit))
• Print: "Semantic extraction: ~N files → X agents, estimated ~Ys"

Step B0 - Check extraction cache first

Before dispatching any subagents, check which files already have cached extraction results:

$(cat graphify-out/.graphify_python) -c "
import json
from graphify.cache import check_semantic_cache
from pathlib import Path

detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
all_files = [f for files in detect['files'].values() for f in files]

cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files)

if cached_nodes or cached_edges or cached_hyperedges:
    Path('graphify-out/.graphify_cached.json').write_text(json.dumps({'nodes': cached_nodes, 'edges': cached_edges, 'hyperedges': cached_hyperedges}))
Path('graphify-out/.graphify_uncached.txt').write_text('\n'.join(uncached))
print(f'Cache: {len(all_files)-len(uncached)} files hit, {len(uncached)} files need extraction')
"

Only dispatch subagents for files listed in graphify-out/.graphify_uncached.txt. If all files are cached, skip to Part C directly.

Step B1 - Split into chunks

Load files from graphify-out/.graphify_uncached.txt. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). When splitting, group files from the same directory together so related artifacts land in the same chunk and cross-file relationships are more likely to be extracted.

Step B2 - Dispatch ALL subagents in a single message

Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose.

IMPORTANT - subagent type: Always use subagent_type="general-purpose". Do NOT use Explore - it is read-only and cannot write chunk files to disk, which silently drops extraction results. General-purpose has Write and Bash access which the subagent needs.

Concrete example for 3 chunks:

[Agent tool call 1: files 1-15, subagent_type="general-purpose"]
[Agent tool call 2: files 16-30, subagent_type="general-purpose"]
[Agent tool call 3: files 31-45, subagent_type="general-purpose"]

All three in one message. Not three separate messages.

Each subagent receives this exact prompt (substitute FILELIST, CHUNKNUM, TOTALCHUNKS, and DEEPMODE):

You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment.
Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble.

Files (chunk CHUNK_NUM of TOTAL_CHUNKS):
FILE_LIST

Rules:
- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2")
- INFERRED: reasonable inference (shared data structure, implied dependency)
- AMBIGUOUS: uncertain - flag for review, do not omit

Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns).
  Do not re-extract imports - AST already has those.
Doc/paper files: extract named concepts, entities, citations. Also extract rationale — sections that explain WHY a decision was made, trade-offs chosen, or design intent. These become nodes with `rationale_for` edges pointing to the concept they explain.
Image files: use vision to understand what the image IS - do not just OCR.
  UI screenshot: layout patterns, design decisions, key elements, purpose.
  Chart: metric, trend/insight, data source.
  Tweet/post: claim as node, author, concepts mentioned.
  Diagram: components and connections.
  Research figure: what it demonstrates, method, result.
  Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS.

DEEP_MODE (if --mode

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [kyopark2014](https://github.com/kyopark2014)
- **Source:** [kyopark2014/agent-skills](https://github.com/kyopark2014/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.