Oss Review

OSS Review via Codex MCP

Get a deep external review of a repository's open-source readiness from Codex MCP with maximum reasoning depth.

Context: $ARGUMENTS

Tool Dispatch (priority order)

1. Official Codex MCP tools (preferred): call mcp__codex__codex for the initial review and mcp__codex__codex-reply for follow-ups. These are the canonical entry points — always try them first.
2. Codex skill helpers: if the Codex MCP tools are unavailable or return an error, invoke the codex:rescue skill as a fallback to delegate the review task through the Codex CLI runtime.
3. Manual fallback: if neither is reachable, record the failure in OSS_REVIEW.md, surface the blocker to the user, and suggest they verify their Codex MCP setup (codex setup && claude mcp add codex -s user -- codex mcp-server).

Never skip step 1 and jump to a fallback. Always attempt the official tool first.

Constants

• REVIEWER_MODEL = gpt-5.4. Use a currently available Codex model; prefer gpt-5.4, gpt-5.3-codex, or gpt-5.2-codex.
• REVIEW_DOC = OSS_REVIEW.md
• READY_THRESHOLD = overall score >= 7/10 with verdict ready or almost

Input Contract

• Primary input: the repository plus any hardening artifacts that already exist:
• OSS_AUDIT.md
• OSS_PLAN.md
• OSS_REFACTOR.md
• OSS_TEST_STRATEGY.md
• OSS_CI.md
• OSS_DOCS.md
• OSS_HARDENING_STATUS.md
• Optional input: explicit publication target, contributor audience, support expectations, or release constraints.
• Default: if hardening artifacts are partial, review the current repository state and clearly label missing evidence.

Output Contract

Create or update OSS_REVIEW.md in the repository root. The review must contain:

1. Repository scope reviewed
2. Overall verdict: ready, almost, or not ready
3. Overall score out of 10
4. Category scorecard:

• onboarding and usability
• correctness and user safety
• maintainability and structure
• testability and automation
• CI and release hygiene
• documentation, licensing, and contributor clarity
• security and responsible maintenance

1. Ranked strengths
2. Ranked weaknesses
3. Minimum fixes required before public release
4. Recommended return stage for each weakness: audit, plan, refactor, tests, ci, or docs
5. Full raw reviewer response, preserved verbatim

Also update OSS_HARDENING_STATUS.md with the latest review score, verdict, and suggested return stage if the review is not yet positive.

Non-goals

• Do not treat style preferences as blocking issues unless they affect maintainability or contributor success.
• Do not claim release readiness without checking tests, docs, and automation evidence.
• Do not overwrite prior review history; append or version it.

Workflow

Step 1: Gather the repository briefing

Before calling Codex MCP, assemble a concise but complete review packet:

• current repo purpose and target users
• entry points and setup commands
• verification commands that currently pass
• hardening artifacts and what they claim was improved
• license, citation, reproduction instructions, and external asset dependencies if applicable
• remaining known gaps or deferred work

Read the highest-signal files first:

• README*
• CONTRIBUTING.md
• package/build manifests
• CI workflows
• test directories
• security/changelog docs if present
• the OSS_*.md hardening artifacts listed above

Step 2: Run the initial external review via official Codex MCP tool

Call the official mcp__codex__codex tool directly. This is the preferred entry point — do not wrap it in Bash or use any intermediate layer.

mcp__codex__codex:
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    You are acting as a senior open-source maintainer and reviewer.

    Review this repository for open-source readiness.

    Repository briefing:
    [paste repo summary, commands, artifacts, and current gaps]

    Please evaluate:
    1. Onboarding and usability
    2. Correctness and user safety
    3. Maintainability and structure
    4. Testability and automation
    5. CI and release hygiene
    6. Documentation, licensing, and contributor clarity
    7. Security and responsible maintenance

    For each category:
    - score it from 1-10
    - explain the most important strengths and weaknesses

    Then provide:
    - an overall score from 1-10
    - overall verdict: Ready / Almost / Not Ready
    - the minimum fixes required before public release
    - a recommended return stage for each fix: audit / plan / refactor / tests / ci / docs
    - a mock maintainer review summary suitable for a PR or launch checklist

    Be direct and practical. Focus on the smallest fixes that materially improve open-source quality.

Save the returned threadId immediately — it is required for all follow-up calls.

If mcp__codex__codex fails (tool not found, connection error, timeout), fall back to codex:rescue skill. If that also fails, record the failure and surface the blocker.

Step 3: Continue with follow-up review via official Codex MCP reply tool

Use mcp__codex__codex-reply (not a new mcp__codex__codex call) with the saved threadId when:

• you need clarification on a weakness
• you want the reviewer to reassess after targeted fixes
• you want a narrower "minimum launchable subset"

mcp__codex__codex-reply:
  threadId: [saved from Step 2]
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    [follow-up question or re-assessment request]

Useful follow-up prompts:

• "Which of these issues are true launch blockers versus post-launch improvements?"
• "What is the smallest fix package that would move this from not ready to almost ready?"
• "Map each weakness to the exact files or docs a maintainer should change next."
• "Re-score the repository now that these fixes were applied: [summary]."

Step 4: Document the result

Append to OSS_REVIEW.md using this structure:

## Review Round N (timestamp)

### Verdict
- Overall score: X/10
- Verdict: ready / almost / not ready

### Category Scorecard
| Category | Score | Notes |
|----------|-------|-------|
| onboarding and usability | 8/10 | ... |
| correctness and user safety | 7/10 | ... |

### Strengths
- ...

### Weaknesses
- P0 / P1 / P2 style ranking with return stage mapping

### Minimum Fixes Before Release
- ...

### Reviewer Raw Response

Full external review

[paste the complete raw response verbatim]

Also record in OSS_HARDENING_STATUS.md:

• latest review score
• latest verdict
• if not ready, the next stage to revisit first

Stop and Return Rules

• If verdict is ready, finish the hardening pass.
• If verdict is almost, either stop with the recorded gaps or do one targeted pass and rerun /oss-review.
• If verdict is not ready, return to the highest-leverage recommended stage and only rerun review after fixes are applied.

Anti-patterns

• Do not ask the external reviewer to judge a repo without a concrete repo briefing and current verification evidence.
• Do not paraphrase or trim the raw reviewer response; preserve it verbatim.
• Do not treat stylistic preferences as release blockers.

Self-check

Before declaring this stage complete, verify:

• [ ] OSS_REVIEW.md exists and contains verdict, overall score, category scorecard, strengths, weaknesses, minimum fixes, return-stage mapping, and raw response.
• [ ] The raw reviewer response is preserved verbatim.
• [ ] OSS_HARDENING_STATUS.md was updated with the latest score, verdict, and next recommended stage.
• [ ] Prior review history was appended to rather than overwritten.

Key Rules

• ALWAYS call the official mcp__codex__codex / mcp__codex__codex-reply tools first; only fall back to codex:rescue if the MCP endpoint is unreachable
• ALWAYS use config: {"model_reasoning_effort": "xhigh"}
• Save and reuse threadId for all follow-up calls within the same review session
• Preserve the full raw reviewer response
• Ask for minimum fixes, not an aspirational roadmap
• Treat missing tests, broken setup, missing docs, missing license or citation path, irreproducible claims, and unsafe release posture as higher severity than polish issues
• Keep the review grounded in repo evidence, not imagined release processes

Failure Handling

• If mcp__codex__codex is unreachable, retry once. If still unavailable, fall back to codex:rescue skill. If neither works, record the failure in OSS_REVIEW.md with the error details and surface the blocker to the user with setup instructions.
• If the reviewer returns an unusable or truncated response, use mcp__codex__codex-reply to request completion on the same thread.
• If the review reveals a foundational gap, return to the recommended stage rather than patching surface issues.
• If prior review history exists, append the new round rather than overwriting it.

Done Criteria

• OSS_REVIEW.md contains an appended review round with verdict, score, scorecard, strengths, weaknesses, minimum fixes, return-stage mapping, and raw response verbatim.
• OSS_HARDENING_STATUS.md records the latest external review score, verdict, and next recommended stage.
• Prior review history remains intact.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: zeyuzhangzyz
• Source: zeyuzhangzyz/open-source-hardening-skills
• License: MIT

Install and usage instructions live in the source repository linked above.