Immersive Spatial Video Production

Immersive Spatial Video Production

This skill is for passive video deliverables that surround or partially surround the viewer: monoscopic 360, stereoscopic 360, VR180, spatial video, dome/wide-FOV video, and headset-first immersive films. It helps an agent make production decisions, write capture and finishing plans, review outputs, and troubleshoot delivery failures without depending on one camera vendor, headset, or generator.

Do not activate this skill for interactive XR applications, OpenXR/WebXR app design, volumetric scene generation, Gaussian-splat/world-model navigation, realtime game-engine builds, or ordinary flat 16:9/9:16 edits unless the requested output explicitly needs spherical, stereo, spatial, wide-FOV, or headset delivery.

Evidence Map

Documented facts below use these sources, verified 2026-07-11 unless a source date is named inline:

• Apple Human Interface Guidelines, "Immersive experiences," https://developer.apple.com/design/human-interface-guidelines/immersive-experiences. Access path: Apple Developer > Design > Human Interface Guidelines > Immersive experiences. Supports the visionOS passthrough, immersion-style, 1.5-meter boundary, movement, permission, attention-cue, and transition-comfort claims in this skill.
• Apple Human Interface Guidelines, "Playing video," https://developer.apple.com/design/human-interface-guidelines/playing-video. Access path: Apple Developer > Design > Human Interface Guidelines > Playing video. Supports the system video player, original aspect ratio, subtitles/transport controls, visionOS comfort, automatic full-immersion warning, thumbnail-track, and RealityKit closed-caption claims in this skill.
• Apple Developer Documentation, "Reading stereoscopic 3D video files," https://developer.apple.com/documentation/avfoundation/mediareadingandwriting/readingstereoscopic3dvideo_files. Access path: Apple Developer > Documentation > AVFoundation > Media reading and writing > Reading stereoscopic 3D video files. Used only for the narrow claim that Apple documents AVFoundation workflows for reading stereoscopic video files.
• Apple Developer Documentation, "Writing spatial video," https://developer.apple.com/documentation/avfoundation/mediareadingandwriting/writingspatial_video. Access path: Apple Developer > Documentation > AVFoundation > Media reading and writing > Writing spatial video. Used only for the narrow claim that Apple documents AVFoundation workflows for writing spatial video.
• YouTube Help: Upload 180- or 360-degree videos; Use spatial audio in 360-degree and VR videos.
• Google Spatial Media Spherical Video V2 RFC and Spatial Audio RFC, plus Spatial Media Metadata Injector v2.1 release notes.
• Khronos OpenXR and W3C WebXR specifications for boundary definition only: they describe interactive XR runtimes, not passive video production deliverables.
• Professional production heuristics are labeled as heuristics, because comfort thresholds vary by headset optics, viewer sensitivity, screen scale, and playback app.

What The Agent Must Decide First

Start by asking what kind of presence the viewer needs, then choose the smallest immersive format that achieves it.

| Format | Use when | Avoid when | Production implications | | --- | --- | --- | --- | | Monoscopic 360 | Place, event, journalism, architecture, documentary context, or location survey where looking around matters more than depth | Close human performance, intimacy, objects within arm's reach, heavy motion | Simpler rig and stitch; weaker scale/depth; hides crew poorly because everything is visible | | Stereoscopic 360 | Viewer should perceive depth throughout a full sphere | Fast action near the rig, many seam crossings, low budget, heavy VFX, viewers may watch on flat screens | Hardest stitch; depth mismatches around seams and poles can be uncomfortable; needs headset QA | | VR180 stereo | Performance, education, travel, sports, product demo, or story content with a natural front stage | Viewer must inspect behind them or understand a full environment | Stronger stereo quality and easier directing; crew can work behind camera; attention stays forward | | Apple-style spatial video | Personal memories, intimate documentary, object-scale moments, headset/photo-library viewing | Need full 180/360 environment or platform-neutral spherical playback | Treat as stereoscopic video with constrained FOV; preserve native metadata and test in target Apple playback path | | Dome or wide-FOV immersive | Planetarium, installation, simulator, or venue projection | Headset distribution is primary | Requires venue mapping, safe horizon, and playback-system tests rather than generic YouTube metadata | | Flat companion cut | Marketing, accessibility, social trailer, editorial review, stakeholders without headsets | It is the only deliverable for an immersive-first commission | Reframe from the spherical master; do not simply crop random viewports without attention design |

Documented fact: YouTube supports upload and playback of 180-degree and 360-degree spherical videos on desktop browsers and YouTube/YouTube VR apps on many VR headsets; YouTube spatial audio supports First Order Ambisonics and First Order Ambisonics with head-locked stereo. Verified 2026-07-11 from YouTube Help.

Production heuristic: prefer VR180 stereo over 360 stereo when the story has a clear front, because it gives the viewer stronger depth and the crew a safe working zone behind camera. Use full 360 only when the viewer genuinely benefits from turning around.

Capture Planning

A useful immersive video brief includes:

• viewer posture: seated, standing, walking only in the real world, or venue-seated;
• intended viewer agency: look-around ambience, front-stage performance, guided attention, or inspection;
• platform priority: Vision Pro, YouTube VR, Quest browser/player, dome, museum player, native app, archival master;
• stereo requirement: mono, stereo 180, stereo 360, spatial video, or both stereo and monoscopic fallbacks;
• audio requirement: mono/stereo, head-locked stereo, FOA ambisonics, FOA plus head-locked narration/music, or platform-specific spatial mix;
• accessibility requirement: captions/subtitles, transcript, audio description, motion-comfort version, and flat fallback;
• rights boundary: people, private property, sensitive location geometry, bystanders, signage, music, and platform terms.

Capture rig decisions:

• Use fewer lenses when possible. Every lens boundary is a seam, and seams are where parallax, exposure, flare, and timing mismatches become visible.
• For 360 mono, prioritize dynamic range, synchronized exposure, and clean lens overlap.
• For stereo 360, prioritize synchronized shutters, stable interaxial geometry, calibration, and subjects kept away from seam zones.
• For VR180 stereo, prioritize lens cleanliness, level horizon, viewer eye-height placement, and a front-stage blocking plan.
• For spatial video, preserve the camera's native stereo relationship and metadata through ingest and export; avoid transcoding steps that flatten stereo to 2D unless intentionally making a companion cut.
• For action, prefer moving the subject within a stable world over moving the rig. When the rig must move, choose smooth pathing, low acceleration, and a visible stable horizon.

Documented fact: Apple HIG says people should be able to choose when to start video and should not be automatically launched into fully immersive video; it also recommends keeping people able to see surroundings during playback and avoiding virtual content that obscures playback controls in fully immersive contexts. Verified 2026-07-11.

Production heuristic: set the camera at or near the viewer's expected eye height. Unmotivated high, low, or tilted placement can make viewers feel like a floating observer rather than present in the scene.

Stitch, Nadir, Poles, And Seams

Treat stitch quality as a story and comfort problem, not only a technical cleanup pass.

Pre-shoot seam strategy:

• Draw seam maps for the chosen rig before blocking actors or action.
• Keep faces, hands, text, product labels, and close foreground objects out of known seam corridors.
• Put seam lines through low-detail, distant, or static areas when possible.
• Control lighting so neighboring lenses do not disagree on exposure or white balance.
• Slate and sync all cameras and recorders; even small temporal offsets can break stitch and stereo comfort.

Post stitch review:

• Check the horizon level before creative stabilization.
• Inspect seams while scrubbing, not only on still frames.
• Review high-contrast lines, moving limbs, reflective objects, smoke, water, and crowds.
• Patch nadir only when the patch is less distracting than the rig footprint. A blurred disk under the viewer may be more honest and comfortable than a warped fake floor.
• Keep pole regions clean, because equirectangular distortion can hide defects in edit view that become obvious in headset playback.

Production heuristic: if a subject must cross a seam, stage the crossing far from the rig and in a low-attention moment. A distant seam crossing is often acceptable; a near-face seam crossing is usually a reshoot or reframing problem.

Stereo Depth And Comfort

Stereo comfort failures are among the fastest ways to make immersive video unusable. Review stereo as early as possible, ideally on the target headset.

Comfort principles:

• Keep the viewer's expected forward direction stable unless a motivated reveal requires otherwise.
• Avoid rapid rotational motion, handheld shake, and acceleration that the viewer's body does not feel.
• Avoid forcing viewers to converge on very near objects for long periods.
• Avoid objects passing extremely close to the camera unless the whole piece is designed around that effect and tested.
• Avoid vertical disparity, left/right exposure mismatches, left/right color mismatches, sync offsets, and stereo window violations.
• For 360 stereo, be extra cautious near seams, zenith, and nadir because stereo correspondence can degrade there.
• For titles and captions, place text at a comfortable apparent distance and keep it stable relative to the viewing context.

Apple documented fact: visionOS immersive experiences use passthrough to keep people connected to physical surroundings; in progressive and full immersion, the system defines an approximately 1.5-meter boundary and increases passthrough/fades the experience as the wearer approaches or passes it. Apple also advises avoiding encouraging movement in progressive or full immersion. Verified 2026-07-11.

Production heuristics:

• Prefer seated or stationary viewing unless the deliverable is intentionally designed for a supervised venue.
• Keep critical action inside the viewer's natural forward field most of the time. Let viewers explore between beats, not during beats that carry plot, safety, or instructions.
• When judging stereo, use a conservative viewer, not the most VR-tolerant person on the team.
• If the camera moves, make the path visually predictable and give viewers stable references such as floor, horizon, architecture, or vehicle frame.

Blocking And Attention Direction

Immersive video cannot assume the editor controls the frame, so direction must be built into staging, audio, lighting, and motion.

Attention cues that usually work:

• actor eyeline and body orientation;
• sound arriving from the area of interest;
• brightness, color contrast, or motion contrast in the intended region;
• foreground-to-background movement that leads the gaze;
• objects entering from peripheral vision slowly enough to be noticed;
• temporary quiet or stillness elsewhere in the sphere;
• repeated spatial geography, so viewers learn where important things happen.

Avoid relying on:

• hard cuts that move the viewer's facing direction without preparation;
• tiny details hidden outside the likely forward field;
• subtitles as the only clue to look somewhere;
• constant action in every direction;
• sudden loud sounds behind the viewer without narrative reason.

Production heuristic: in immersive story cuts, treat every shot as a stage picture plus an invitation. The viewer should know where to look, but should not feel punished for looking elsewhere.

Stabilization And Motion

Stabilization must preserve orientation truth while reducing discomfort.

• Level the horizon unless a stylized or physical reason justifies roll.
• Correct jitter before creative reframing.
• Avoid excessive yaw smoothing that lags behind a moving platform and makes the world feel detached.
• If walking footage is unavoidable, test whether a shorter cut, locked forward view, or flat companion cut is more humane.
• In 360, stabilization can reveal blank or patched areas at poles and nadir; inspect after stabilization, not before only.
• For headset playback, verify that metadata and pose are still correct after stabilization and export.

Apple documented fact: Apple HIG recommends smooth, predictable transitions between immersive styles and warns that sudden transitions can be disorienting or uncomfortable. Verified 2026-07-11.

Spatial And Ambisonic Audio

Audio often does more attention work than the picture.

Use these layers deliberately:

• Ambisonic bed: environment, crowd, room tone, ambience, diegetic space.
• Directional objects or panned elements: attention cues and visible sound sources.
• Head-locked stereo: narration, music, accessibility narration, or UI-like content that should not rotate with the viewer.
• Standard stereo fallback: non-headset delivery and flat companion cuts.

Documented fact: YouTube accepts First Order Ambisonics and First Order Ambisonics with Head-Locked Stereo for 360-degree and VR videos. Its upload guidance says to create the 360/VR video with spatial audio, run the latest Spatial Media metadata tool, and upload the result. Verified 2026-07-11.

Documented fact: Google's Spatial Audio RFC defines an MP4 SA3D box for spatial audio metadata. For first-order periphonic ambisonics, four ambisonic components are required, and the RFC describes ACN channel ordering and SN3D normalization. Verified 2026-07-11.

Production heuristics:

• Record a clean ambisonic bed at camera position, plus isolated lavs/booms for dialogue. The ambisonic recording alone is rarely enough for intelligible speech.
• Keep narration head-locked unless there is a strong story reason for the narrator to occupy a world position.
• Do not over-pan every sound. Stable ambience plus a few meaningful spatial cues is usually more comfortable than a busy sphere.
• Always check audio rotation in headset: front should sound front, left should sound left, and head-locked elements should remain attached to the viewer.

Titles, Captions, Subtitles, And Accessibility

Immersive text must be readable, comfortable, and discoverable without becoming a visual trap.

Documented facts:

• Apple HIG Playing video says the system video player provides transport controls that include subtitle options, and RealityKit video playback supports closed captions. Verified 2026-07-11.
• Apple advises restricting extra metadata/overlays so they do not obscure media playback, and in fully immersive contexts warns against virtual content obscuring playback or transport controls. Verified 2026-07-11.

Caption and title guidance:

• Prefer captions that follow the viewer's current forward view or sit in a consistent comfortable lower region, depending on platform capability.
• For speaker identification, combine caption labels with spatial audio and blocking; do not make viewers spin to find every speaker.
• Avoid placing captions at the equirectangular poles or across stitch seams.
• Keep caption depth consistent and comfortable in stereo deliverables.
• Use high contrast without pure bright panels that burn into OLED displays or dominate the scene.
• Provide transcript and flat captioned fallback when platform caption suppor

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: calesthio
• Source: calesthio/generative-media-skills
• License: MIT
• Homepage: https://github.com/calesthio/OpenMontage

Install and usage instructions live in the source repository linked above.