Img2threejs

img2threejs — Image to procedural Three.js

Rebuild the object visible in a reference image as a code-only procedural Three.js model, gated by a staged sculpting pipeline and an AI-vision self-correction loop. This is reconstruction-by-code, not photogrammetry, mesh extraction, or downloaded art packs.

Use Codex-native image reading, an already-enabled browser tool, the project preview, or a user-supplied screenshot wherever this workflow calls for agent vision or a browser.

Runtime and paths

Resolve SKILL_ROOT to the directory containing this SKILL.md. When Codex exposes PLUGIN_ROOT, this skill is at ${PLUGIN_ROOT}/skills/img2threejs; otherwise use the skill's discovered absolute path. Run every bundled script by absolute path from SKILL_ROOT and keep generated specifications, renders, and models in the user's project or an approved output directory, never in the installed plugin.

Use the active Python 3.10+ executable: py -3 or python on Windows, and python3 or the configured environment's interpreter on macOS. Pass arguments as an array and do not build a shell command. In command examples, ` means that selected executable and means the resolved absolute SKILL_ROOT`. The harness itself uses only the standard library.

The local specification index never writes into the installed plugin. Use --cache-root or AI_GAME_STUDIO_CACHE_DIR; otherwise a bounded OS user-cache directory is used. Generated specifications, code, renders, and evidence still belong in the approved project/output directory.

The built-in pixel tools decode compatible PNG directly. Common JPEG/TIFF/BMP inputs may be converted through the fixed native Windows System.Drawing or macOS /usr/bin/sips route. A non-native ImageMagick fallback is allowed only when AI_GAME_STUDIO_IMAGEMAGICK_MANIFEST points to a confirmed JSON record containing its absolute executable, SHA-256, version, reviewed license, HTTPS source URL, and plan digest. Unsupported formats must stop with a conversion request; never run an executable selected only from PATH.

This plugin vendors upstream v1.4.3 under Apache-2.0. Read [UPSTREAM.json](../../UPSTREAM.json) and [LICENSE](../../LICENSE) before redistributing changes.

Core Promise

Sculpt from a photo, in order — never one-shot a mesh:

1. Run "/forge/next.py" first. It reports the current unlocked pass, exact next command, and unmet acceptance criteria.
2. Validate the image is a suitable 3D target (grimoire/intake/validation_rubric.md).
3. Assess object class + complexity, then write a qualityContract before any code.
4. Spec it: component hierarchy, materials, lighting, pivots, sockets, action anchors.
5. Build pass-by-pass from blockout → structure → form → material → lighting → interaction → optimization.
6. Verify each pass with a screenshot compared against the reference; fail a pass if an identity-defining feature is wrong even when the global score looks fine.

State explicitly when output is approximate/stylized/low-poly. A single image cannot reveal hidden sides or guarantee exact geometry — say so instead of faking confidence.

Transparency and Process Debugging (Critical — from Bowie Knife reconstruction)

The problem: When the user cannot tell what was done or where something went wrong, they cannot debug the process. Over-claiming (reporting success when features still don't match) destroys trust and makes iterative improvement impossible.

Rule: Be transparent + don't over-claim. State exactly what changed each pass, with evidence, and name what still doesn't match:

• After each pass, explicitly list what changed: "Updated guard shape to extend left edge from -0.56 to -0.48 for handle overlap"
• Provide evidence: reference the specific values, coordinates, or parameters that changed
• Name what still doesn't match: "Handle silhouette traced but still flat plane (no Z palm-swell), procedural crosshatch not reference's exact dot-grid knurl"
• Explain why a change was made: "Extended guard left edge because handle ends at X=-0.42 and guard ended at X=-0.20, causing visual gap"
• Never claim a feature is "done" when it's only "improved" — use precise language
• When a gate passes but visual inspection shows issues, explain the limitation: "2D gate passed (fidelity 0.83) but three-quarter render shows blade reads as toy (no grind wedge) — 2D gates are blind to 3D realism"

The user needs to be able to debug the process, not just the output. If something is wrong, they should be able to trace which decision led to the error and correct it. Opaque processes force restarts; transparent processes enable refinement.

Required Inputs

• one image path / screenshot / URL / attached image (if missing or unreadable, ask)
• intended use: prop, game object, hero render, playable/destructible object, animation rig

(default: real-time browser prop with interactive performance)

• for a CS2 request, an authoritative classification record (family/subtype and evidence refs) or

an explicit request for the user/vision provider to supply one; heuristic detection alone is not enough to select a geometry adapter

Download a URL to an approved local input path only through a separately reviewed network step. The forge accepts local image files; it does not treat an arbitrary URL as permission to fetch or upload content.

The Loop (scripts do enforcement; agent vision does judgment)

Run scripts through their absolute paths under /forge/.... Pure Python 3.10+ stdlib, no pip installs. Full flags: grimoire/scripts.md. Never let a script score visuals — that is the agent's job.

1. Analyze the image first (agent vision, before any script): work the layered observation

protocol in grimoire/intake/image_analysis.md — identify/classify, decompose macro→meso→micro, map part relationships, name materials in PBR terms, list identity-defining features, and flag what the single view hides. Observation before inference; controlled 3D vocabulary; 3D object-space not 2D image-space. This is generic for any subject and feeds every field below. Then probe local images: forge/stage1_intake/probe_image.py (metadata only, not a visual check). 1a. Local Spec Search — after image analysis and before writing or refining a spec, local evidence is a pipeline stage, not an optional memory lookup, whenever the request needs domain-specific anatomy, PBR, wear, geometry, runtime, or physics specifications. The pre-spec command automatically runs BM25, chooses cs2 for CS2 targets and core_3d otherwise, and writes a localSpecSearch evidence bundle into the assessment: "/forge/stage2_spec/new_pre_spec_assessment.py" "Name" --image --out assessment.json. Add observed terms with repeatable --spec-query ""; use --collection only when the automatic collection choice is insufficient. new_sculpt_spec.py --assessment carries that bundle into the final spec, including snippets, source_refs, and evidence_refs. For extra focused retrieval, the direct CLI remains available: "/forge/stage1_intake/search_specs.py" "" --collection --limit 3 --snippet-chars 250 --json. For CS2, include English/Vietnamese variants, for example --spec-query "safety ring vòng ngón" or search_specs.py "roughness độ nhám" --collection cs2. Expand queries with object names, component names, material/finish terms, behavior terms, and bilingual aliases; retry focused alternatives when the first result is incomplete. Build the spec from returned evidence and do not invent domain specs when local evidence exists. Search caches are local/generated only; preserve JSONL records and source provenance rather than replacing them with cache output. 1b. CS2 intake manifest — for a CS2 request, create and validate cs2-intake.json before pre-spec authoring. Run admission and probing for every source view, record the heuristic signal as non-authoritative evidence, attach the classification record, resolve the supported family, and choose route independently from exactnessTier. Missing classification, insufficient coverage, or a contradictory high-confidence class is request-input; unsupported families do not continue into spec generation.

1. Pre-Spec Assessment Gate — classify + score complexity + write the quality contract:

forge/stage2_spec/new_pre_spec_assessment.py "Name" --image --complexity --out assessment.json. Rules: grimoire/intake/quality_contract.md. Set objectClass.primaryDomain (object | character | hybrid) and fill the seeded detailInventory (its targetMinDetails scales with complexity). Supported CS2 knife skins: always pass --cs2, which defaults the complexity tier to ultra-complex (targetMinDetails 16) — the finish/wear/hardware is the item, so CS2 is held to the top fidelity bar; targetMinDetails never drops below the 9 floor even if downgraded by hand. Author procedural GEOMETRY (blade/guard/grip profiles) but make the FINISH a de-lit reference-crop PROJECTION, not a procedural finish material — projecting the photo's own pixels is what reaches reference fidelity for patterned skins (Doppler/Gamma/Marble/Fade), and is what the v1.3 baseline demos do; a procedural finish for a patterned skin reads visibly wrong against the reference. Take the projection path in step 2c (it generalizes from characters to any reference-matched surface). Procedural finish is the fallback ONLY when live view-dependent response matters more than matching this one reference. Finish routes + rulebook: grimoire/build/cs2_finishes.md; optional exact-texture acquisition: grimoire/intake/cs2_texture_acquisition.md. 2b. Detail inventory (do not skip for detailed subjects) — scan zones and enumerate every identity-defining small detail (gloss, bevel, fasteners, linework, contours, stains): forge/stage1_intake/build_detail_inventory.py --mode grid-3x3 --out-dir --out di.json. Each detail MUST map to a component.localFeatures or material.localOverrides entry — never prose only. Taxonomy + 3D-term recipes: grimoire/intake/detail_inventory.md. 2c. Projection-first fidelity (characters AND reference-matched surfaces — supported CS2 knife skins, decals, painted patterns) — when the goal is matching a specific reference's surface, put the photo's own pixels on the mesh instead of approximating them procedurally. This is the single biggest fidelity lever; a procedural material for a patterned surface is the #1 reconstruction failure. Recipe (grimoire/character/likeness_maximization.md — its two levers, align-mesh+camera and project-the-photo, generalize past characters): solve the camera (stage1_intake/solve_camera_pose.py → referenceCamera), de-light the reference so it is free of baked lighting (stage1_intake/delight_albedo.py, hard requirement — this is what makes projection safe, not the flat-lit icon), then project the de-lit crop onto the mesh and bake it into UVs (stage3_build/bake_projected_texture.py --mesh-id ). For a CS2 skin the mesh is the procedural blade/guard/grip you author in the spec, and the projected de-lit crop IS the finish (front + back from the two views) — no procedural Doppler material. For characters, first capture landmarks (stage1_intake/extract_landmarks.py --out anatomy.json), fill preSpecAssessment.anatomy, route grimoire/character/reconstruction.md. A single view cannot show hidden sides — report per-region confidence and request more views when it matters.

1. Author the spec from the assessment:

forge/stage2_spec/new_sculpt_spec.py "Name" --image --assessment assessment.json --manifest cs2-intake.json --out object-sculpt-spec.json. Replace generic starter featureReviewTargets with the object's real identity-defining systems (≤5 critical, ≤3 important per pass); for characters add anatomy-proportion, face-landmark-placement, pose-silhouette, outfit-and-palette. Use 3D-graphics terms only (grimoire/glossary/3d_vocabulary.md), never "nice/smooth/shiny". Classify every component's topologyClass/topologyRationale per grimoire/intake/surface_topology.md before picking a primitive — this is what prevents a continuous organic form from being picked as a box.

1. When material fidelity matters and a source image exists, analyze each material's finish then

extract reference PBR evidence, both per crop (crop the correct region — verify the crop is on the part you think it is):

• forge/stage1_intake/analyze_texture.py --spec spec.json --material-id --in-place

classifies the finish (gem-metal | gemstone | painted-metal | worn-composite | brushed-steel | plastic), extracts the gradient palette, and writes doc-grounded MeshPhysicalMaterial scalars (metalness/roughness/clearcoat/transmission/ior/anisotropy/envMapIntensity) onto the material. Recipes + Three.js texture/PBR rules (colorSpace, CanvasTexture/DataTexture, height→normal) live in grimoire/build/threejs_texture_reference.md. Rule of thumb: solid albedo for flat paint, real reference crop for patterned finishes (doppler/quartz/hydro-dip/camo).

• forge/stage1_intake/extract_pbr_evidence.py --out-dir --material-id --target-threshold 0.7.

Confidence forge/stage3build/generatethreejs_factory.py object-sculpt-spec.json --out src/createObjectModel.ts (generator is pass-gated: a future --pass-id fails until prior passes are reviewed continue`).

1. Render the current pass in a browser/preview, capture a screenshot at a review viewpoint.
2. Package one side-by-side sheet, then inspect it with agent vision:

forge/stage4_review/make_comparison_sheet.py --reference --render --out cmp.png --json.

1. Record the review (overall + per-layer + per-feature scores + decision):

forge/stage4_review/append_review.py object-sculpt-spec.json --pass-id --fidelity --action --summary "..." --render-screenshot --comparison-image cmp.png --ai-vision-score --layer-scores-json '{...}' --feature-reviews-json --in-place. For the CS2 knife path, also attach the versioned report with --cs2-review-json cs2-review.json --review-scene-json forge/tests/fixtures/knife_review_scene.json. A failed family, painted-region, projection-coverage, critical-detail, or orbit gate blocks continue even when the global score passes. See docs/cs2/review-gates.md.

1. Sync pipeline state after manual review edits:

forge/stage3_build/orchestrate_passes.py sync object-sculpt-spec.json --in-place.

CS2 image-matched rule

For a CS2 item, the target is observable agreement between the supplied image and the rendered item: silhouette, proportions, edge profile, hardware layout, coating colour, pattern placement, wear, roughness response, and camera framing. Every decision must be traceable to evidence or be labelled as an approximation.

The initial CS2 family boundary is knife only. Pistol, rifle, SMG, sniper, heavy, glove, and unknown knife subtypes must stop with unsupported-family or unsupported-subtype; they must not receive the knife component tree as a generic fallback.

Layer contract

Pass these records between layers. Do not copy an informal vision description into the next stage:

| Layer | Owns | Must emit | Must not decide alone | | --- | --- | --- | --- | | Intake | view validity and technical evidence | role, path/hash, resolution, coverage, duplicate status, admission verdict | item identity from aspect ratio or filename | | Classification | semantic identity | family, subtype, confidence, evidence refs, provider/version, timeout state | geometry or finish parameters | | Identity | skin/name/paint metadata | precedence, resolved values, ambiguity candidates, proven

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: frabcd
• Source: frabcd/codex-ai-game-studio
• License: MIT
• Homepage: https://frabcd.github.io/codex-ai-game-studio/

Install and usage instructions live in the source repository linked above.