Twin Test

Twin Test -- Adversarial Clone Fidelity Check

A blind taste test where the clone generates responses to the same contexts as real user messages, then a discriminator identifies which is real and which is the clone. Specific style corrections feed back into the user model.

How It Works

1. Sample -- Pull 3-5 real sent messages from memory (the "ground truth")
2. Generate -- For each message, generate a clone response to the same context
3. Discriminate -- A separate agent compares pairs and identifies the real message
4. Score -- Calculate fidelity (% of times the discriminator is fooled)
5. Correct -- Extract specific style corrections from discriminator feedback

Backend tools (use these, don't improvise)

• twin_test_sample (mcp__nomos-think__twin_test_sample) -- pulls N real

messages + their contexts to test against. Use this for the Sample phase.

• twin_test_record (mcp__nomos-think__twin_test_record) -- after you've

discriminated each pair, pass results (per pair: true = the judge spotted the real message, false = fooled). It computes the fidelity score with the documented formula and PERSISTS it to the DB for the trend.

• twin_test_history (mcp__nomos-think__twin_test_history) -- the stored

score history. Use for /twin-test score (do not keep scores in chat memory).

Commands

• /twin-test -- Run a full twin test session (3-5 message pairs)
• /twin-test score -- Show fidelity score history

Session Protocol

When the user invokes /twin-test, follow this exact protocol:

Phase 1: Sample Selection

1. Call memory_search with category "exemplar" to find high-quality real messages
2. If no exemplars, search for sent messages in memory (source: "sent", direction: "outgoing")
3. Select 3-5 diverse messages that cover different contexts (work, casual, technical)
4. For each message, extract the conversational context (what was the user replying to?)

Phase 2: Clone Generation

For each sampled message:

1. Reconstruct the context: who was the user talking to, what was the conversation about
2. Generate YOUR response to the same context, using all personality data (user model, style profiles, exemplars, values)
3. Try your absolute best to match the user's voice -- this is the test

Important: generate responses BEFORE showing the user any results. Do not look at the real message while generating.

Phase 3: Discrimination

For each message pair (real + clone):

1. Present both messages to yourself in randomized order (A and B)
2. Act as a discriminator: which message is the real user and which is the clone?
3. Explain your reasoning: what specific markers distinguish the messages?
4. Note the confidence of your assessment

Phase 4: Results

Present a summary:

Twin Test Results
=================

Fidelity Score: XX% (X/Y pairs where discriminator was fooled)

Pair 1: [context summary]
  Real: "..." (correctly/incorrectly identified)
  Clone: "..."
  Discriminator notes: [what gave it away]

Pair 2: ...

Style Corrections:
- [specific corrections based on discriminator feedback]

Phase 5: Corrections

For each pair where the discriminator correctly identified the clone:

1. Extract what was different (tone, word choice, length, punctuation, emoji usage, formality)
2. Store these as corrections in the user model at confidence 0.8
3. Update style profiles if specific style markers were identified
4. Tell the user what you learned

Phase 6: User Review

Ask the user:

• "Were my assessments accurate? Did I identify the right messages as real?"
• "Any of these clone responses that were actually close to what you'd say?"
• Accept corrections and store them

Important Rules

• Blind generation -- generate clone responses BEFORE comparing to real messages
• Honest assessment -- if you can't tell which is real, say so (that's a good fidelity sign)
• Specific corrections -- "slightly too formal" is better than "didn't match"
• Store corrections at 0.8 -- adversarial testing is high-quality signal
• Track over time -- compare against previous twin-test scores
• Diverse samples -- try to test across different contexts and platforms
• Don't game it -- the goal is honest assessment of where the clone falls short

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: project-nomos
• Source: project-nomos/nomos
• License: MIT
• Homepage: https://projectnomos.dev

Install and usage instructions live in the source repository linked above.