Structsense

> Skill version 0.6.1. Two things carry across every mode. Concept mapping is mandatory and tool-only: the pipeline cascades local hybrid → BioPortal → ask the user for an alternate URL → hard-stop, and items carrying concept_mapping_provenance: "llm_knowledge" are demoted to unmapped, because a hallucinated IRI is worse than an honest gap. ABCD/HBCD mode (rule 16) maps a paper's own wording to the NBDC/NDA dictionary using the instrument, respondent, metric and release it states, keeps only what that study did itself, and emits a cross-paper synthesis whose every row carries its provenance. Legacy outputs can be brought up to spec via python -m scripts.normalize_result --input --llm-model (idempotent). See [CHANGELOG.md](CHANGELOG.md).

StructSense Skills — structured information extraction

A reusable methodology for turning unstructured text and PDFs into clean, schema-conformant JSON, with optional ontology grounding and quality scoring. The patterns here are model-agnostic: they work with Claude, GPT, Gemini, Pi, or any local model.

When to invoke this skill

Trigger when the user asks to:

• Extract named entities + key terms (NER) from biomedical, neuroscience, or scientific text.
• Pull resources out of papers — tools, datasets, models, benchmarks, leaderboards.
• Convert a document into a target JSON schema (e.g. ReproSchema, Croissant, a custom schema the user supplies).
• Map extracted terms to ontologies (BioPortal, OLS, OBO, BTO, CL, UBERON, NCBITaxon, MESH, …).
• Score or judge the quality of an existing extraction.
• Process a long document that needs chunking and parallel runs.
• Extract ABCD / HBCD study content from publications — which variables a study used, the constructs behind them, the models specified, the findings reported — and compare across papers: where is there consensus, where divergence, which variables are consistently mediators or moderators. → references/abcd-extraction.md.

The core pattern

Four cooperating roles, run sequentially. Each role's output is the next role's input. Any role can use a different model.

┌───────────┐   raw text     ┌────────────┐   extracted   ┌───────┐   aligned   ┌──────────────┐
│ EXTRACTOR │ ──────────────►│ ALIGNMENT  │──────────────►│ JUDGE │────────────►│ HUMAN FB     │
│ (LLM)     │                │ (LLM+tool) │               │ (LLM) │             │ (optional)   │
└───────────┘                └────────────┘               └───────┘             └──────────────┘
   strict JSON                 + ontology fields           + judge_score          + corrections
                               + provenance                + remarks              + revised JSON

| Stage | Job | Reads | Writes | |---|---|---|---| | Extractor | Find entities/resources/fields. Output strict JSON. | raw text | items with entity/name, label/type, sentence, start, end (etc.) | | Alignment | Map each item to an ontology IRI. | extractor output | adds ontology_id, ontology_label, ontology, concept_mapping_provenance (tool or llm_knowledge) | | Judge | Score quality of each item (0–1). | alignment output | adds judge_score, remarks | | Human feedback | Apply corrections from a human reviewer. | judge output + user feedback | revised JSON |

You can run any subset — see references/pipeline-pattern.md.

Who runs the LLM stages — read this before asking for an API key

The four roles above say what runs, not who runs it. There are two modes, and picking the wrong one is the most common way a run stalls before it starts.

| | Host-model mode (the default when an agent is reading this) | Framework mode | |---|---|---| | Who is the extractor / judge | you, the model reading this file | scripts/pipeline.py, calling out over HTTP | | Where it applies | Claude Code, Codex CLI, Claude Desktop, Pi, any agent session | batch jobs, cron, CI, an MCP server, a script | | LLM API key | none — there is no API to call | required (OPENROUTER_API_KEY / ANTHROPIC_API_KEY / OPENAI_API_KEY) | | --extractor / --judge (pipeline.py) | do not pass them — nothing to point at | required | | --llm-model (normalize_result.py) | do pass it, set to your own model id — it is a provenance label, not a call | pass the extractor model | | How the prompt is used | read prompts/.md and follow it yourself | passed to the provider by llm_client.py |

If you are an agent reading this, you are in host-model mode. Read the extractor prompt and produce the JSON yourself, then use the scripts for the deterministic work — mask_pass.py, group_by_entity.py, normalize_result.py, stats.py, iri_validation.py. None of those call an LLM. So the whole pipeline runs with no LLM API key at all, and asking the user for one is a bug, not diligence.

Switch to framework mode only when the user explicitly wants it: a headless/scheduled run, or a different model than the host (cheaper extraction, a local Ollama, a model you can't be). Then --extractor and a key are genuinely required.

Two keys that are not LLM keys, and are needed in either mode:

• BIOPORTAL_API_KEY — the concept-mapping tool (rule 15's cascade). Free, and

the only key that ever matters for a host-model run. If mapping falls through to BioPortal and this is unset, ask for this by name — never as "an API key".

• SEMANTIC_SCHOLAR_API_KEY, and similar service keys — optional rate-limit lifts.

When you do need to ask, name the exact variable and what breaks without it. "This needs an API key" is the ambiguous phrasing that sends users hunting for an OpenRouter account they don't need.

Quick decision flow

1. What kind of extraction?

• Entities + key terms (NER) → load references/ner-extraction.md, then pick the extractor prompt by domain:
• General-domain text (news, finance, biographies, generic web pages, mixed text) → prompts/extractor-ner-general.md.
• Neuroscience text — broad (behavior + systems + cellular + molecular + computational) → prompts/extractor-ner-neuroscience.md.
• CNS-cell-focused text (cell atlases, patch-seq, scRNA-seq cell typing, BICCN-style cell census — anything where cell types + markers + morphology + ephys are the subject) → prompts/extractor-ner-cns-cells.md, plus references/cell-annotation-conventions.md if the output will be scored against a human gold standard (specificity types, nested spans, coordinated ids — the conventions that make the difference between a real error and a format mismatch).
• Tools / datasets / models / benchmarks → load references/resource-extraction.md and prompts/extractor-resource.md.
• User has a target JSON schema → load references/structured-extraction.md and prompts/extractor-structured.md.
• ABCD / HBCD variables, models, findings, or cross-paper synthesis → load references/abcd-extraction.md and prompts/extractor-abcd.md. This mode has its own verifier and its own hard rules (see rule 16); it is not a variant of NER. Single PDF or a directory in bulk; every run emits JSON + Markdown + Turtle.

1. Want exhaustive recall? (almost always yes for NER) → after pass-1 extraction, run the mask-recall pass with prompts/mask-recall-pass.md + scripts/mask_pass.py. Optionally also run mask-verify (prompts/mask-verify-pass.md) for per-item label sanity. See references/ner-extraction.md → "Two-pass strategy: mask-mode".

2b. Biomedical text? Enable the HuggingFace NER ensemble. Pass --ner-profile biomedical_broad (or cns_cells / pharmacology / genetic / clinical / minimal / all) to run specialist models alongside the LLM extractor. Every mention carries a source_model field; the grouped view records consensus_count (how many models agreed). See references/ner-models.md. Skip the ensemble for non-biomedical text or when transformers isn't installed.

1. Need ontology mapping? → load references/ontology-mapping.md. Default cascade: local hybrid at http://localhost:8000 (verify at /docs) → BioPortal → ask the user for an alternative URL → skip alignment only if declined. Don't hardcode the URL; the port and host vary across deployments.
2. Long document (>10 pages or > model context)? → load references/chunking-strategy.md. Chunk → run extractor in parallel → merge → run downstream stages.
3. Need quality scoring? → load prompts/judge.md.
4. Multiple models for cost? Use the cheapest capable model for extraction (often a small open model), a stronger model for alignment if you don't have a mapping tool, and a fast model for judging. See references/model-selection.md.

Hard rules

These prevent the most common failures.

1. Strict JSON output, no markdown fences. Every prompt must include "Output strict JSON only. No prose. No markdown fences." in the system message. Set temperature: 0 for extraction and alignment.
2. Extract EXHAUSTIVELY. For NER, emit every occurrence of every mention as a distinct item with its own start/end. Never deduplicate by surface form. A multi-page neuroscience paper should yield hundreds to thousands of entity items, not a few hundred. If yield feels low, run the mask-recall pass (prompts/mask-recall-pass.md + scripts/mask_pass.py) — typical recovery is +30–80%.
3. Preserve fields downstream. Alignment, judge, and human-feedback stages add fields. They never remove existing fields and never re-key existing items.
4. Record provenance. Every mapped item carries concept_mapping_provenance: "tool" | "llm_knowledge". Never hide where a mapping came from.
5. Chunk and merge for inputs longer than the model's context window (or > 25,000 chars for safety on 128k models). Always re-merge by stable identifiers (sentence + char span, or item id).
6. Don't invent placeholders. The agent communication contract is: extractor input is the raw text; alignment input is the extractor's JSON; judge input is the alignment's JSON. Pipe outputs cleanly — don't re-wrap or paraphrase between stages.
7. Validate before returning. Parse the JSON; if parsing fails, repair-then-retry (see references/json-output-discipline.md). Validate against the task's JSON schema in schemas/.
8. Always emit a stats block. Every final result must embed a stats block at the top level (totals, label histogram, alignment provenance, judge score buckets, per-stage elapsed times) and print a human-readable summary to stderr. Use scripts/stats.py. This is the answer to "did the run do what it was supposed to?" — a healthy NER run on a paper has hundreds-to-thousands of entity mentions and mentions_per_unique > 1. A summary with 230 mentions and mentions_per_unique ≈ 1 is the symptom of surface-form deduplication; re-run with the mask-recall pass and double-check no upstream step is collapsing duplicates.
9. Final-result filename convention. When writing the result to disk, name it _final.json (e.g. paper.pdf → paper_final.json, note.txt → note_final.json). Honor an explicit --out only when the user provides one. The reference helper is scripts/pipeline.py::default_output_path.

9b. More than one document? Deliver the corpus view too, not just N per-paper files. In framework mode this is automatic: pipeline.py --input (or a repeated --input) runs each paper, writes each _final.json, and then merges them into corpus_synthesis.{json,md} — auto-detected from the input count, exactly as abcd_extract decides on its synthesis, with --no-synthesize / --synthesize to override. In host-model mode you are the loop, so nothing runs it for you: after the last paper, run python -m scripts.merge_corpus --out /corpus_synthesis yourself — a directory works, no glob needed, and it skips anything that looks like a previous roll-up so a re-run cannot fold its own output back in. Per-paper _final.json stays the authoritative record of raw mentions; the roll-up adds one canonical row per entity across every paper, which documents it appears in, and where papers disagree about its ontology id. Handing back a directory of per-paper JSON and leaving the user to reconcile it is an unfinished deliverable: the questions a corpus is for ("which cell types does this collection talk about", "which mappings conflict") cannot be answered from any single file. The index is grouped, not concatenated — pass --include-mentions only if the raw union is genuinely wanted.

1. Concept-mapping cascade — and you MUST probe before declaring unavailable.

Default mapper is the local hybrid service at http://localhost:8000. Before saying "no mapper available" you MUST run at least one probe in your current runtime: ``bash curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8000/docs ` If the probe returns 200, USE the mapper. The real API schema is **{ "max_results": N, "text": [{"text": "...", "context": "..."}] }** — NOT terms:[...]. See prompts/alignment-via-http.md` for a turnkey curl + jq pipeline.

• On connection refused: try BioPortal (BIOPORTAL_API_KEY).
• On further failure: ask the user for an alternative URL (ports 8001 / 8080 / 9000 / reverse-proxied paths are common) — do not give up silently.
• Only after the user declines should you skip alignment (concept_mapping_provenance: "skipped").
• If your runtime can't reach the user's localhost (claude.ai web app, Anthropic Skills hosted, ChatGPT cloud), say so explicitly and direct the user to the MCP bridge or tunnel options in connecting/mcp-server.md. Don't pretend the service is unreachable when the user has it running — be explicit that the runtime is the constraint.

1. Always tag source_model provenance. Every entity item carries source_model (HF model id like d4data/biomedical-ner-all, or llm_ner: for LLM-extracted items). The grouped view (entities_grouped[]) lists all contributing models per entity and the consensus_count. Never strip or merge these fields. See references/ner-models.md.
2. Document metadata lives at the top, not on every entity. paper_title / doi / source_path go into a top-level source_metadata block — ONCE per run. paper_location (section / page / paragraph) stays per-entity because it varies. Never repeat document-level metadata on hundreds of items.
3. Always emit both entities[] (raw, one per occurrence) and entities_grouped[] (canonical, with merged sentences from every location). The raw list is the authoritative record for exhaustive extraction; the grouped list is what downstream consumers navigate by. Use scripts/group_by_entity.py:attach_grouped_views — it's automatic in pipeline.run().
4. Canonical-shape guarantee via normalizer. scripts/normalize_result.py runs automatically before every save and produces the canonical shape regardless of what the LLM emitted: top-level source_metadata, stripped per-entity paper_title/doi, tagged source_model on every item, entities_grouped attached, stats embedded. Idempotent — safe to run on already-canonical results. It's also exposed as a CLI to fix legacy result files in place. If you ever see legacy output, do not panic and do not edit by hand — run python -m scripts.normalize_result and the file is brought up to spec.
5. Concept mapping is MANDATORY and TOOL-ONLY. Zero hallucination.

• The pipeline never silently skips alignment. Default cascade: local hybrid (http://localhost:8000, verify at /docs) → BioPortal (BIOPORTAL_API_KEY) → ask the user for an alternate URL → hard-stop with a clear error. OLS is no longer in the auto-cascade (it has no gene coverage); the user must opt in explicitly via --allow-ols-fallback.
• concept_mapping_provenance: "llm_knowledge" is **

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: sensein
• Source: sensein/agent_skills
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.