# Forge 3d Picking

> >

- **Type:** Skill
- **Install:** `agentstack add skill-nebulavenus-forge-gpu-forge-3d-picking`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [Nebulavenus](https://agentstack.voostack.com/s/nebulavenus)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Zlib
- **Upstream author:** [Nebulavenus](https://github.com/Nebulavenus)
- **Source:** https://github.com/Nebulavenus/forge-gpu/tree/main/.claude/skills/forge-3d-picking

## Install

```sh
agentstack add skill-nebulavenus-forge-gpu-forge-3d-picking
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# 3D Picking

Add GPU-based object picking to an SDL GPU project. This skill covers two
picking methods (color-ID and stencil-ID), GPU-to-CPU data transfer with
`SDL_DownloadFromGPUTexture`, transfer buffer readback, and stencil outline
selection highlighting.

Based on [GPU Lesson 37](../../../lessons/gpu/37-3d-picking/).

## When to use

- You need to identify which object the user clicked on in a 3D scene
- You want pixel-perfect selection without CPU-side ray casting
- You need to highlight selected objects with a visible outline
- You want to read data back from the GPU to the CPU

## Color-ID picking pipeline

Color-ID picking renders each object with a unique flat color to an offscreen
texture, then reads back the single pixel under the cursor to identify the
object. This is the recommended approach — it is portable, supports up to
65,535 objects, and the RGBA readback has a consistent byte layout across all
GPU backends.

### Index-to-color encoding

```c
/* Convert object index to a unique RGB color.
 * Index 0 maps to ID 1 (background is (0,0,0) = no object). */
static void index_to_color(int index, float *r, float *g, float *b)
{
    int id = index + 1;
    *r = (float)((id >> 0) & 0xFF) / 255.0f;
    *g = (float)((id >> 8) & 0xFF) / 255.0f;
    *b = 0.0f;
}

/* Decode a read-back pixel back to an object index. Returns -1 for background. */
static int color_to_index(Uint8 r, Uint8 g, Uint8 b)
{
    (void)b;
    int id = (int)r | ((int)g  ds_bpp) ? color_bpp : ds_bpp;
SDL_GPUTransferBuffer *pick_readback = SDL_CreateGPUTransferBuffer(device, &xfer_ci);
```

### Copy pass

Download a single pixel from the source texture into the transfer buffer:

```c
SDL_GPUCopyPass *copy = SDL_BeginGPUCopyPass(cmd);

SDL_GPUTextureRegion src_region;
SDL_zero(src_region);
src_region.texture = id_texture;  /* or depth-stencil for stencil-ID */
src_region.x = (Uint32)pick_x;
src_region.y = (Uint32)pick_y;
src_region.w = 1;
src_region.h = 1;
src_region.d = 1;

SDL_GPUTextureTransferInfo dst_info;
SDL_zero(dst_info);
dst_info.transfer_buffer = pick_readback;
dst_info.offset = 0;

SDL_DownloadFromGPUTexture(copy, &src_region, &dst_info);
SDL_EndGPUCopyPass(copy);
```

### Wait, map, and decode

After submitting the command buffer, wait for GPU completion before reading:

```c
if (!SDL_SubmitGPUCommandBuffer(cmd)) {
    SDL_Log("SDL_SubmitGPUCommandBuffer failed: %s", SDL_GetError());
    return;
}
if (!SDL_WaitForGPUIdle(device)) {
    SDL_Log("SDL_WaitForGPUIdle failed: %s", SDL_GetError());
    return;
}

void *pixel_data = SDL_MapGPUTransferBuffer(device, pick_readback, false);
if (pixel_data) {
    Uint8 *bytes = (Uint8 *)pixel_data;
    int picked = color_to_index(bytes[0], bytes[1], bytes[2]);
    selected_object = (picked >= 0 && picked = object_count) {
    stencil = bytes[0];  /* fallback for reversed byte order */
    picked = (int)stencil - 1;
}
```

## Selection outline with stencil

Highlight the selected object using the two-pass stencil outline technique from
[Lesson 34](../../../lessons/gpu/34-stencil-testing/):

### Outline write pipeline

```c
/* Pass 1: draw selected object, write stencil marker */
pi.depth_stencil_state.enable_stencil_test = true;
pi.depth_stencil_state.front_stencil_state = (SDL_GPUStencilOpState){
    .fail_op       = SDL_GPU_STENCILOP_KEEP,
    .pass_op       = SDL_GPU_STENCILOP_REPLACE,
    .depth_fail_op = SDL_GPU_STENCILOP_KEEP,
    .compare_op    = SDL_GPU_COMPAREOP_ALWAYS,
};
pi.depth_stencil_state.back_stencil_state =
    pi.depth_stencil_state.front_stencil_state;
pi.depth_stencil_state.write_mask   = 0xFF;
pi.depth_stencil_state.compare_mask = 0xFF;
```

### Outline draw pipeline

```c
/* Pass 2: draw scaled-up object where stencil != marker */
pi.depth_stencil_state.enable_depth_test  = false;
pi.depth_stencil_state.enable_depth_write = false;
pi.depth_stencil_state.enable_stencil_test = true;
pi.depth_stencil_state.front_stencil_state = (SDL_GPUStencilOpState){
    .fail_op       = SDL_GPU_STENCILOP_KEEP,
    .pass_op       = SDL_GPU_STENCILOP_KEEP,
    .depth_fail_op = SDL_GPU_STENCILOP_KEEP,
    .compare_op    = SDL_GPU_COMPAREOP_NOT_EQUAL,
};
pi.depth_stencil_state.back_stencil_state =
    pi.depth_stencil_state.front_stencil_state;
pi.depth_stencil_state.write_mask   = 0x00;  /* don't modify stencil */
pi.depth_stencil_state.compare_mask = 0xFF;
pi.rasterizer_state.cull_mode = SDL_GPU_CULLMODE_NONE;
```

### Outline rendering

```c
#define STENCIL_OUTLINE  200  /* high value to avoid picking collisions */
#define OUTLINE_SCALE    1.04f

/* Outline pass: LOAD color + depth, CLEAR stencil */
SDL_GPUColorTargetInfo outline_color = {
    .texture  = swapchain_tex,
    .load_op  = SDL_GPU_LOADOP_LOAD,
    .store_op = SDL_GPU_STOREOP_STORE,
};
SDL_GPUDepthStencilTargetInfo outline_ds = {
    .texture          = main_depth,
    .load_op          = SDL_GPU_LOADOP_LOAD,
    .store_op         = SDL_GPU_STOREOP_DONT_CARE,
    .stencil_load_op  = SDL_GPU_LOADOP_CLEAR,
    .stencil_store_op = SDL_GPU_STOREOP_DONT_CARE,
    .clear_stencil    = 0,
};

SDL_GPURenderPass *pass = SDL_BeginGPURenderPass(
    cmd, &outline_color, 1, &outline_ds);

/* Step 1: draw object normally, write stencil marker */
SDL_BindGPUGraphicsPipeline(pass, outline_write_pipeline);
SDL_SetGPUStencilReference(pass, STENCIL_OUTLINE);
/* ... push uniforms, draw selected object ... */

/* Step 2: draw scaled-up object with NOT_EQUAL stencil */
SDL_BindGPUGraphicsPipeline(pass, outline_draw_pipeline);
SDL_SetGPUStencilReference(pass, STENCIL_OUTLINE);

mat4 outline_model = mat4_multiply(
    mat4_translate(object_position),
    mat4_scale_uniform(object_scale * OUTLINE_SCALE));
/* ... push outline uniforms (solid color), draw ... */

SDL_EndGPURenderPass(pass);
```

## Key API calls

- `SDL_CreateGPUTransferBuffer()` with `DOWNLOAD` usage — CPU-readable staging
  buffer for GPU-to-CPU data transfer
- `SDL_DownloadFromGPUTexture()` — copy a texture region into a transfer buffer
  (runs inside a copy pass)
- `SDL_WaitForGPUIdle()` — block until all submitted GPU work completes
- `SDL_MapGPUTransferBuffer()` / `SDL_UnmapGPUTransferBuffer()` — access the
  downloaded pixel data on the CPU
- `SDL_SetGPUStencilReference()` — set per-draw stencil reference value for
  stencil-ID picking and outline rendering

## Common mistakes

- **Using the swapchain format for the ID texture** — The swapchain format
  varies by platform and may be sRGB, which distorts the ID color encoding.
  Always use `R8G8B8A8_UNORM` for the ID texture.
- **Missing depth buffer on the ID pass** — Without its own depth buffer, the
  ID pass renders all objects without occlusion. Background objects overdraw
  foreground objects, producing incorrect pick results.
- **Reading transfer buffer before GPU completion** — The download is
  asynchronous. Reading the transfer buffer before `SDL_WaitForGPUIdle` (or
  fence completion) returns stale or garbage data.
- **Forgetting to clear stencil for the outline pass** — If the outline pass
  reuses stencil values from the scene pass (stencil-ID mode), the outline
  comparison will fail. Clear stencil at the start of the outline render pass.
- **Stencil reference collision** — If the outline marker value (e.g., 200)
  overlaps with stencil-ID object indices (1-N), the outline will not render
  correctly for those objects. Use a high marker value well above your object
  count.
- **Running the ID pass every frame** — The ID pass re-renders the entire scene.
  Running it every frame doubles the draw call count. Only run it on click frames
  unless you specifically need hover detection.

## Cross-references

- [GPU Lesson 37 — 3D Picking](../../../lessons/gpu/37-3d-picking/)
  for the full walkthrough
- [GPU Lesson 34 — Stencil Testing](../../../lessons/gpu/34-stencil-testing/)
  for the stencil outline technique
- [GPU Lesson 36 — Edge Detection](../../../lessons/gpu/36-edge-detection/)
  for alternative stencil patterns
- [GPU Lesson 06 — Depth & 3D](../../../lessons/gpu/06-depth-and-3d/)
  for depth buffer fundamentals

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Nebulavenus](https://github.com/Nebulavenus)
- **Source:** [Nebulavenus/forge-gpu](https://github.com/Nebulavenus/forge-gpu)
- **License:** Zlib

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-nebulavenus-forge-gpu-forge-3d-picking
- Seller: https://agentstack.voostack.com/s/nebulavenus
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.