Obsidian Kb

/obsidian-kb — Build a Project Knowledge Base

Analyze any project's structure and build a custom knowledge base: tiered context packs for LLM consumption + an Obsidian wiki for human browsing. Inspired by Karpathy's LLM Knowledge Base workflow, adapted for LLM-first consumption.

Usage

/obsidian-kb init                # Analyze project, propose architecture
/obsidian-kb build               # Generate compiler, commands, run initial compile
/obsidian-kb rebuild             # Re-run init + build (for when project structure changes)

---

Phase 1: init — Analyze & Design

Step 1: Scan the Project

Scan the project directory tree. Respect .gitignore if it exists. Collect:

1. Data files by format:

• .jsonl — JSON Lines (line-delimited records)
• .json — JSON documents or arrays
• .yaml / .yml — YAML documents
• .csv / .tsv — Tabular data
• .toml — Structured configuration
• .md with YAML frontmatter — Structured markdown

1. Project signals — detect what kind of project this is:

• package.json or tsconfig.json → JavaScript/TypeScript software project
• setup.py or pyproject.toml or requirements.txt → Python software project
• Cargo.toml → Rust project
• go.mod → Go project
• .bib or papers/ or references/ → Research/academic project
• JSONL files with name + description + domain fields → Knowledge graph project
• clients/ or projects/ directories → Consulting/portfolio project
• docs/ or documentation/ → Documentation-heavy project
• data/ with structured files → Data-centric project

1. Existing compilation — check if data/compiled/ or data/kb-architecture.yaml already exist. If so, note them and ask if the user wants to rebuild or update.

Display: "Scanned [N] directories, found [M] data files across [K] formats. Project type: [detected type]."

Step 2: Sample and Classify Knowledge Atoms

For each data file found:

1. Read 3-5 sample records (first 3-5 lines for JSONL, first 3-5 entries for JSON arrays, etc.)
2. Identify what each record represents — its knowledge atom type:

• What is the primary identifier field? (name, title, id, path, etc.)
• What is the description/content field?
• What fields create relationships to other records? (references, links, tags, categories)
• What grouping dimension exists? (domain, category, module, type, directory)

For software projects without structured data files, the knowledge atoms come from the code structure:

• Components/modules (from src/ directory structure)
• API endpoints (from route definitions)
• Architecture decisions (from docs/adr/ or decisions/ directories)
• Configuration (from config files)
• Dependencies (from package manifests)

For markdown-heavy projects, extract frontmatter fields as the schema.

Display a table:

Source: data/knowledge-graph/frameworks.jsonl
  Format: JSONL (22 records)
  Atom type: framework
  Key field: name
  Description: description
  Group field: domain
  Relationship fields: entangled_with, related_frameworks
  Sample: "Knowledge Gradient Model" (intelligence_systems)

Source: data/reading-list/index.jsonl
  Format: JSONL (50 records)
  Atom type: reading
  Key field: title
  Description: core_arguments
  Group field: domain
  Relationship fields: framework_connections
  Sample: "Thinking in Systems" (intelligence_systems)

Step 3: Map Relationships

For each pair of knowledge atom types, identify:

1. Explicit references: Field A in type X contains names/IDs from type Y

• Example: concept.related_frameworks → framework.name

1. Shared groupings: Both types share a grouping field with the same values

• Example: framework.domain = concept.domain

1. Hierarchical containment: Type X is a child/component of type Y

• Example: API endpoint belongs to a module

Build a relationship map:

framework  concept (via related_frameworks)
framework  reading (via framework_connections)
framework  framework (via entangled_with)
concept -- framework (shared domain)
signal -> framework (text mention)

Step 4: Design Tier Structure

Based on the atoms and relationships, design a three-tier compiled context pack structure:

Tier 0 — Index (always loaded, ~1000-1500 tokens)

• One-line entry per knowledge atom across ALL types
• Purpose: routing. "What exists in this knowledge base?"
• Output: data/compiled/index.md

Tier 1 — Group Context (loaded per operation, ~3000-6000 tokens each)

• One file per group value (domain, category, module, directory)
• Contains: all atoms in that group with full descriptions, cross-references, relationship summaries
• Purpose: working context for a specific area
• Output: data/compiled/[group_dimension]/[group_value].md
• If no natural grouping exists: use a single "overview" file per atom type

Tier 2 — Entity Deep Context (loaded on demand, ~1000-2000 tokens each)

• One file per entity in the richest atom type (the type with most relationships)
• Contains: the entity + summaries of all connected entities + history + related sources
• Purpose: deep work on a specific item
• Output: data/compiled/[atom_type]/[slug].md
• Only generate Tier 2 for atom types with rich interconnections. Simple lists don't need deep context pages.

Design the wiki structure in parallel:

• One wiki section per atom type
• Wikilinks between related atoms: [[section/slug|Display Name]]
• Frontmatter tags for filtering in Obsidian
• Graph color groups: one color per section

Step 5: Propose Architecture

Present the full architecture to the user:

=== Knowledge Base Architecture Proposal ===

Project: [project name from directory or package.json]
Type: [detected type]

Sources:
  [list of source files with atom types and record counts]

Tier Structure:
  Tier 0 (Index): [what it contains]
  Tier 1 ([group dimension]): [N groups] — [group names]
  Tier 2 ([entity type]): [N entities]

Wiki Structure:
  wiki/
    [section 1]/ — [atom type] ([count] pages)
    [section 2]/ — [atom type] ([count] pages)
    ...
    index.md — Map of Content

Estimated pages: [total]
Estimated compile time: fast ( Open Vault > [project]/wiki

Commands installed:
  /kb — query, search, compile
  /lint-kb — health check

Run /lint-kb to check knowledge base health.

---

Phase 3: rebuild — Re-analyze

When the user runs /obsidian-kb rebuild:

1. Re-scan the project (new sources may have been added)
2. Compare against existing data/kb-architecture.yaml
3. Show what changed (new sources, removed sources, schema changes)
4. Propose updated architecture
5. On approval, regenerate compiler and recompile

---

Design Principles

1. The LLM is both producer and consumer. Context packs are the primary output. The Obsidian wiki is a human-readable export.

1. Architecture spec is the contract. kb-architecture.yaml bridges analysis and generation. Users can edit it. It documents what the KB tracks.

1. Freshness over rebuilding. The compiler checks source timestamps. Slash commands check freshness.json. Recompile only when sources change.

1. The compiler is project-specific code, not a template instantiation. You write the compiler fresh each time, tailored to the project's actual schema and relationships. The BrunoTwin compiler is your reference for structure, but the content is unique per project.

1. Obsidian conventions matter. Wikilinks use [[path/slug|Display Name]] format. Frontmatter uses standard YAML. Tags are lowercase. Graph coloring uses path: queries. These make the wiki work well in Obsidian without plugins.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: chiKeka
• Source: chiKeka/obsidian-kb
• License: MIT

Install and usage instructions live in the source repository linked above.