Dogfood

Dogfood Session

A dogfood session exercises hyalo against real knowledgebases to find bugs, verify fixes, assess UX, and discover improvement opportunities. Sessions should feel like a real user working with the tool — not a mechanical checklist. Be curious, follow interesting leads, try edge cases that occur to you naturally.

Phase 1: Prepare

1. Build from source: cargo build --release
2. Find the last dogfood report to establish the baseline:

``bash hyalo find --tag dogfooding --sort date --reverse --limit 1 --format text `` Read it to learn: which version was tested, which bugs were found, what was verified.

1. Discover what changed since then: Use git log --oneline from that report's date

to HEAD. Look for merged iteration branches (iter-N/...). For each, read the iteration file in the knowledgebase to understand what was implemented: ``bash hyalo find --tag iteration --property "date>=YYYY-MM-DD" --sort date --format text ` Then hyalo read --format text` for each to understand the features, bug fixes, and UX changes. These are your priority test targets.

1. Check prior dogfood reports for open bugs to re-verify:

``bash hyalo find --tag dogfooding --sort date --reverse --limit 5 --format text `` Read the most recent reports and collect any bugs marked as found (not yet verified fixed).

1. Pick target knowledgebases: Use a mix of:

• Own KB (hyalo-knowledgebase/, ~250 files) — well-structured, has schemas/views
• MDN Web Docs (../mdn/files/en-us/, ~14K files) — large, stress-tests performance
• GitHub Docs (../docs/content/, ~3.5K files) — complex nested YAML frontmatter
• VS Code Docs (../vscode/, ~1K files) — mid-size, different structure

For external KBs, check they exist first (ls ../mdn/files/en-us/ > /dev/null 2>&1). Create a snapshot index for large vaults (500+ files).

Phase 2: Exercise the tool

This is the creative part. Don't follow a rigid script — think about what a real user would do with hyalo and try it. That said, make sure every session covers these categories:

New feature verification

Test the features you discovered in Phase 1 from the iteration files and git log. For each new feature or changed command, don't just check the happy path — try edge cases, wrong inputs, combinations with other flags. The iteration files describe what was implemented and often include acceptance criteria that suggest good test scenarios.

Bug regression testing

Re-verify bugs from the most recent dogfood report(s). Use the exact commands or scenarios that originally exposed them. Mark each as STILL FIXED, REGRESSED, or PARTIALLY FIXED.

Creative exploration

This is where you go beyond verification. Try things like:

• Unusual filter combinations (--property + --tag + BM25 + --section)
• Edge-case inputs (empty strings, special characters, very long queries)
• Commands on files with unusual frontmatter (nested objects, empty values, unicode)
• --jq queries to build ad-hoc reports or dashboards
• Views that chain with additional CLI flags
• mv on files with many inbound links
• lint --fix --dry-run on messy external KBs
• Schema validation on write (--validate or validate_on_write)
• Performance comparisons (indexed vs non-indexed, small vs large vault)
• links fix on KBs with broken links
• Anything that occurs to you as you go — follow your curiosity

UX assessment

Pay attention to:

• Are error messages helpful? Do they suggest what to do?
• Are hints useful and relevant?
• Is --format text output scannable?
• Do help texts match actual behavior?
• Any friction points or confusing behavior?

Performance spot-checks

Time a few representative commands on each KB used:

time hyalo find --limit 1                    # baseline parse speed
time hyalo find "some query"                 # BM25 search
time hyalo summary                           # full vault scan
time hyalo find --property status=completed  # structured filter

Compare with prior reports if available. Flag regressions > 2x.

Phase 3: Write the report

Save to hyalo-knowledgebase/dogfood-results/ with this naming and frontmatter:

---
title: "Dogfood vX.Y.Z — "
type: research
date: YYYY-MM-DD
status: active
tags: [dogfooding, ...]
related:
  - "[[dogfood-results/previous-report]]"
  - "[[iterations/relevant-iteration]]"
---

Report structure

Organize findings into clear sections. Use this general shape, but adapt as needed:

# Dogfood vX.Y.Z — 

Brief context: binary version, which KBs tested, file counts.

## New Feature Verification
### Feature Name (iter-NNN) — WORKING / BROKEN / PARTIAL
Description of what was tested and results.

## Bug Regression Testing
### BUG-N: Description — STILL FIXED / REGRESSED
Commands tested, results observed.

## Bugs Found
### BUG-N: Description (SEVERITY)
Steps to reproduce, expected vs actual, impact.

## UX Issues
### UX-N: Description (SEVERITY)
What's confusing or could be better.

## What Worked Well
Highlight things that are genuinely good — performance wins, helpful errors, etc.

## Performance
Table of timings if measured.

Severity levels: HIGH (data loss, wrong results), MEDIUM (confusing behavior, workaround exists), LOW (cosmetic, minor friction).

After writing the report

Use hyalo set to ensure the frontmatter is correct. Add related links to relevant iterations or prior dogfood reports.

Tips for good dogfood sessions

• Follow the hints. Hyalo outputs drill-down suggestions — use them like a real user would.
• Don't skip external KBs. Many bugs only surface at scale or with unusual frontmatter.

The own KB is too well-structured to catch everything.

• Note what's NOT there. Missing features, things you wished you could do — these are

just as valuable as bugs.

• Be specific. Include exact commands, exact error messages, exact output. Vague reports

like "search felt slow" aren't actionable.

• Test across KBs. A command that works on the own KB might fail on MDN due to scale or

on GitHub Docs due to nested YAML.

• Check data quality. Use lint, find --broken-links, find --orphan to surface

issues in the KB itself — this is valuable dogfooding too.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ractive
• Source: ractive/hyalo
• License: MIT

Install and usage instructions live in the source repository linked above.