Tribunal

Tribunal

[](./CHANGELOG.md) [](./LICENSE) [](./CHANGELOG.md) []() []() []() []()

[](https://www.star-history.com/#davccavalcante/tribunal&type=timeline&legend=top-left)

> Governed verification and synthesis for Massive Intelligence (IM) outputs. Run configurable Thinker, Worker, and Verifier panels, deterministic evaluators, and a model-as-judge over an injectable client, fold them into a verdict under a policy you define, and sign and seal that verdict so it can be audited, all on your own infrastructure with your own keys.

Tribunal is the trust layer for model outputs, delivered as a zero-runtime-dependency NPM library that runs in your process, not someone else's. Agent outputs are not reliable by default: models hallucinate, one agent's hallucination becomes the next agent's ground truth, and IM-written code ships defects. The market answer so far is either to leave verification to the developer or to hand it to a closed endpoint that verifies internally but opaquely, where you cannot configure the verifier, cannot audit the verdict, and cannot prove anything to a reviewer. Tribunal takes the third path: explicit, configurable verifier roles; mixture-of-agents cross-verification with a quorum you set; deterministic evaluators alongside a model-as-judge; and a governed verdict that is cryptographically signed and sealed into a tamper-evident log. You decide what "passed" means, and you get a receipt.

Core promise: zero required runtime dependencies, two-line setup, an injectable judge client so the verification logic stays provider-agnostic and testable offline, a seven-level severity ladder with a policy you control, mixture-of-agents panels with configurable quorum, deterministic evaluators (latency, schema, regex, exact-match) plus a model-as-judge, Ed25519-signed verdicts and a tamper-evident SHA-256 audit chain over the Web Crypto API, a node-free core that runs in Node, edge runtimes, and the browser, ESM plus CJS dual distribution, and a framework-agnostic tool adapter that binds to any MCP server, OpenAI-compatible endpoint, or the Vercel AI SDK with no hard dependency.

---

Why Tribunal

Verifying a model output is a trust problem with a governance constraint. You want the strongest possible signal that an answer is correct, complete, and safe, without giving up control over how that signal is produced or the ability to prove it later. The two reflexive answers both fall short. Leaving verification to each developer scatters ad-hoc regexes and one-off "LLM as a judge" prompts with no shared contract and no audit trail. Outsourcing to a closed endpoint that verifies internally gives you a verdict you cannot configure, cannot inspect, and cannot sign. Tribunal uses the correct primitive: configurable roles and evaluators over a judge pool you own, folded into a signed, sealed verdict under a policy you define, on infrastructure you control.

What sets it apart from a closed verifier:

• Configurable and auditable. You define the roles, the evaluators, the rubric, the quorum, and the severity at which a verdict fails. The verdict carries its findings and the provenance of every contributor. Nothing about the judgment is hidden.
• Signed verdicts. Every verdict can be signed with Ed25519 over its canonical bytes, so anyone with the public key can prove a verdict was issued by the holder of the private key and was not altered.
• Tamper-evident trail. Sealed verdicts chain into a SHA-256 hash chain; altering any past entry breaks every hash after it, and verifyChain pinpoints the break.
• Mixture-of-agents cross-verification. A panel of judges deliberates under a quorum you choose (majority, unanimous, any, or at-least-N). Diverse judges catch failure modes a single judge misses.
• Deterministic evaluators plus model-as-judge. Cheap offline checks (latency, schema validity, regex, exact-match) run alongside a configurable model-as-judge, and fold into one governed verdict.
• You define "passed". A seven-level severity ladder (info, low, minor, moderate, major, critical, blocker) plus an optional minimum score; you set the threshold that fails the build.
• Provider-agnostic by construction. The core never speaks a provider API. You inject a judge, an OpenAI-style chat function, a Vercel AI SDK call, or the built-in deterministic heuristic judge, and the same verification logic runs against any of them.
• Local-first and sovereign. It runs in your process, on your infrastructure, with your keys, including in the EU. No regional block, no vendor in the middle of every verdict.

Compared to a long-running agent runtime such as Hermes Agent, which does verification through a kanban swarm and a model-as-judge gated on acceptance criteria but keeps only a plain, unsigned event log, Tribunal is the narrow, deep, embeddable layer that adds the missing piece: a configurable cross-verification quorum and a cryptographically signed, auditable verdict. The two compose; Tribunal can be the verification-and-synthesis layer inside any agent runtime.

---

Install

pnpm add @takk/tribunal
# or: npm install @takk/tribunal
# or: yarn add @takk/tribunal
# or: bun add @takk/tribunal

Node >= 20. Ships ESM and CJS with types. Zero required runtime dependencies.

---

Quickstart

import { createHeuristicJudge, defineRubric, schemaEvaluator, Tribunal } from "@takk/tribunal";

const rubric = defineRubric({
  id: "answer-quality",
  passThreshold: 0.6,
  criteria: [
    { id: "addresses-task", description: "addresses the refund question", weight: 2, mustInclude: ["refund"] },
    { id: "no-placeholder", description: "no leftover placeholder", weight: 1, mustAvoid: ["todo", "tbd"] },
  ],
});

const tribunal = new Tribunal({
  // deterministic checks that fold into the same verdict as the panel
  evaluators: [schemaEvaluator({ schema: { type: "object", required: ["action"] }, failSeverity: "critical" })],
  // mixture-of-agents cross-verification under a quorum you choose
  panel: {
    rubric,
    quorum: "majority",
    judges: [createHeuristicJudge({ id: "v1" }), createHeuristicJudge({ id: "v2" }), createHeuristicJudge({ id: "v3" })],
  },
  // you define what "passed" means
  policy: { failAtOrAbove: "major", minScore: 0.5 },
});

const verdict = await tribunal.tryCandidate({
  id: "agent-output-1",
  content: '{"action":"refund","note":"issue a refund to the customer"}',
});

console.log(verdict.outcome); // "pass" | "fail" | "inconclusive"
console.log(verdict.severity); // worst finding on the seven-level ladder
console.log(verdict.score); // aggregate score in [0, 1]
console.log(verdict.provenance.contributors); // how many evaluators and judges weighed in

The built-in createHeuristicJudge is fully deterministic and makes no network calls, so the whole verification runs offline with reproducible numbers. Swap in a real model-as-judge with one line (next section).

---

Use a real provider

Tribunal never speaks a provider API. You inject a judge; the verification logic stays identical.

OpenAI-compatible chat:

import { judgeClientFromChat } from "@takk/tribunal/adapter";

const judge = judgeClientFromChat({
  id: "gpt-judge",
  chat: async (messages) => {
    const res = await openai.chat.completions.create({ model: "gpt-5.2", messages });
    return res.choices[0]?.message?.content ?? "";
  },
});

const tribunal = new Tribunal({ panel: { rubric, judges: [judge] } });

Vercel AI SDK:

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { judgeClientFromVercel } from "@takk/tribunal/adapter";

const judge = judgeClientFromVercel({
  id: "vercel-judge",
  generateText: ({ system, prompt }) => generateText({ model: openai("gpt-5.2"), system, prompt }),
});

The adapter builds a Verifier prompt from the rubric, asks for strict JSON, and parses it back into a typed reply, recovering per-criterion scores when the model returns them. Mix several judges, even across providers, into one panel for genuine cross-verification.

---

Judge a single candidate against a rubric

When you only need a model-as-judge over one output, use the judge directly:

import { createHeuristicJudge } from "@takk/tribunal";

const judge = createHeuristicJudge();
const reply = await judge.judge({ candidate: "applied the fix and added a regression test", rubric });
console.log(reply.passed, reply.score, reply.criteria);

---

Evaluators and panels

Evaluators are cheap, deterministic checks. Each returns a typed evaluation that folds into the verdict:

• latencyEvaluator({ maxMs }) passes when the candidate was produced within a time budget.
• schemaEvaluator({ schema }) passes when the content parses as JSON and matches a minimal, dependency-free shape spec.
• regexEvaluator({ pattern, mustMatch }) passes when the content matches (or, inverted, does not match) a pattern.
• exactMatchEvaluator({ expected, normalizeWhitespace, caseInsensitive }) passes when the content equals an expected string.

Panels are mixture-of-agents cross-verification. Several judges deliberate over the same candidate and their votes fold under a quorum:

import { deliberatePanel } from "@takk/tribunal/panel";

const result = await deliberatePanel(candidate, {
  rubric,
  quorum: { atLeast: 2 }, // "majority" | "unanimous" | "any" | { atLeast: n }
  judges: [judgeA, judgeB, judgeC],
});
console.log(result.passes, result.needed, result.passed, result.agreement);

---

Severity and policy: you define "passed"

Every finding carries a severity on a seven-level ladder, ordered from least to most serious:

info  <  low  <  minor  <  moderate  <  major  <  critical  <  blocker

A verdict's severity is the worst of its findings. Your policy decides the outcome:

const policy = {
  failAtOrAbove: "major", // any finding at or above this severity fails the verdict
  minScore: 0.7,          // optional: also fail if the aggregate score drops below this
};

This is the heart of the package: the closed alternative decides "passed" for you and hides why. Tribunal makes the definition explicit, configurable, and inspectable.

---

Governed synthesis with provenance

Judge several candidate answers for one task and synthesize the final answer, recording every candidate considered:

const { verdicts, synthesis } = await tribunal.tryCase(
  {
    id: "deploy-plan",
    prompt: "Write a deploy plan.",
    candidates: [
      { id: "draft-a", content: "just push to prod" },
      { id: "draft-b", content: "step 1 build, step 2 deploy, with a rollback step" },
    ],
  },
  { strategy: "best-passing" }, // "best-passing" | "highest-score" | "unanimous-pass"
);

console.log(synthesis.chosen); // "draft-b"
console.log(synthesis.content); // the chosen answer, never invented

Synthesis never fabricates content: the chosen answer is always one of the candidates Tribunal actually judged.

---

Signed, tamper-evident verdicts

Sign a verdict with Ed25519 and seal it into a hash chain, in one call:

import { generateSigningKeyPair } from "@takk/tribunal/signing";
import { verifyChain } from "@takk/tribunal/audit";

const keyPair = await generateSigningKeyPair();
const sealed = await tribunal.sealedVerdict(candidate, keyPair);

// the signed envelope carries the public key needed to verify it
console.log(sealed.signed.algorithm); // "Ed25519"

// the chain is tamper-evident: altering any entry breaks every hash after it
console.log((await verifyChain(sealed.chain.toArray())).valid); // true

Or do it piecewise with signVerdict / verifyVerdictSignature and an AuditChain you own. Both the signature and the seal use the Web Crypto API, so the core stays node-free and runs anywhere.

---

Entry points

The root export is node-free. Every concern is also a focused subpath so you import only what you use.

| Import | What it gives you | |---|---| | @takk/tribunal | The full node-free surface: Tribunal, evaluators, judge, panel, rubric, verdict, synthesis, signing, audit, roles, adapter, types. | | @takk/tribunal/tribunal | The Tribunal facade only. | | @takk/tribunal/evaluators | latencyEvaluator, schemaEvaluator, regexEvaluator, exactMatchEvaluator, runEvaluators, EvaluatorRegistry. | | @takk/tribunal/judge | createHeuristicJudge, createConstantJudge, judgeEvaluator. | | @takk/tribunal/panel | deliberatePanel, quorumNeeded. | | @takk/tribunal/rubric | defineRubric, scoreRubric, summarizeRubric. | | @takk/tribunal/verdict | buildVerdict, renderVerdict, verdictLine, severity policy. | | @takk/tribunal/synthesis | governedSynthesis. | | @takk/tribunal/signing | generateSigningKeyPair, signVerdict, verifyVerdictSignature. | | @takk/tribunal/audit | AuditChain, sealVerdict, verifyChain. | | @takk/tribunal/roles | Thinker / Worker / Verifier definitions and prompt scaffolds. | | @takk/tribunal/adapter | judgeClientFromChat, judgeClientFromVercel, createVerifyTool, parseJudgeReply. | | @takk/tribunal/node | Node-only JSON loaders (the only entry that imports node: builtins). | | @takk/tribunal/edge | The node-free barrel, explicit for edge runtimes and the browser. |

---

A tool for a non-human entity

Expose Tribunal as a framework-agnostic tool any agent or MCP client can call:

import { createVerifyTool } from "@takk/tribunal/adapter";

const tool = createVerifyTool(tribunal);
// { name: "tribunal_verify", description, parameters (JSON Schema), handler }

const result = await tool.handler({ candidate: "the answer to verify", expected: "the answer to verify" });
// a JSON-safe governed verdict, ready to hand back to the model

Verification as a service: any non-human entity (a MAIC agent, an NHE, or any tool-calling model) can submit an output and receive a governed, auditable verdict.

---

CLI

# Verify a case (several candidates) against a rubric panel; exit 10 if it fails.
npx @takk/tribunal verify case.json --rubric rubric.json --quorum majority --fail-at major

# Judge a single candidate file against a rubric; exit 10 if it fails.
npx @takk/tribunal judge answer.txt --rubric rubric.json

# Generate an Ed25519 signing key pair (print, or --out a file).
npx @takk/tribunal keygen --out keys.json

# Verify a tamper-evident audit chain; exit 20 if broken.
npx @takk/tribunal audit-verify chain.json

# Render a verdict and verify its signature; exit 21 on a bad signature.
npx @takk/tribunal verdict signed.json

Exit codes: 0 ok / pass, 2 usage or input error, 10 verdict failed, 20 broken audit chain, 21 invalid signature. The CLI runs offline; the panel uses the deterministic heuristic judge.

---

How it works, in one paragraph

A candidate output is run through the configured deterministic evaluators and, if present, a panel of model-as-judge verifiers that deliberate under a quorum. Each evaluator and the panel produce a finding with a severity on the seven-level ladder. buildVerdict folds them into a single governed outcome under your policy: it fails if any finding lands at or above your failAtOrAbove severity, or if the aggregate score drops below your minScore. The verdict records the provenance of every contributor. It can then be signed with Ed25519 over its canonical (sorted-key) JSON and sealed into a SHA-256 hash chain whose every entry binds to the previous hash, so the log is tamper-evident. The whole pipeline is verification logic over an injectable JudgeClient, so it is provider-agnostic and runs offline in tests with the deterministic heuristic judge.

---

What the examples show

Each example in [examples/](./examples) runs against the built dist with real numbers:

• verify-candidate evaluators plus a judge panel produce a governed verdict with findings and provenance.
• **cros

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: davccavalcante
• Source: davccavalcante/tribunal
• License: Apache-2.0
• Homepage: https://davccavalcante.github.io/tribunal/

Install and usage instructions live in the source repository linked above.