Looker To Sigma

Looker → Sigma Conversion

> Windows / first run — run the environment doctor before anything else: > bash scripts/doctor.sh (macOS/Linux/Git Bash) or powershell -ExecutionPolicy Bypass -File scripts\doctor.ps1 (Windows). > It checks Ruby/Python/Node/bash and flags the Python "Store stub" + CRLF with exact fixes. Details: refs/environment.md.

Preflight the workbook spec before POST (mandatory)

Before POSTing any workbook spec, run ruby scripts/lib/preflight_lint.rb — it exits 1 with a precise message on the two migration-killer bugs: a table with aggregate columns + dimensions but no groupings (renders raw detail rows), and a malformed control (missing id/controlId/controlType or nesting value fields under a value object instead of flat, a non-double-nested source, or a list control wired to neither source nor filters — a filters-only list control is valid). Fix every violation first — never POST past it, and never conclude a feature is "unsupported" from an Invalid kind error (it means the inner fields are wrong). Verified shapes: sigma-workbooks controls.md / tables.md.

Phase 0a — Choose where to build (ask first when no destination given)

Don't silently land the migrated data model + workbook in an auto-picked folder. If the user didn't supply a destination (no --folder and no SIGMA_FOLDER_ID), ASK before building:

1. python3 scripts/pick_destination.py list → { workspaces, folders (editable, with parentName), myDocuments }
2. Let the user pick ONE: a workspace (its id lands content in the workspace root),

an existing folder, My Documents (when non-null — null for service tokens), or create a new folder: python3 scripts/pick_destination.py create --name "" [--parent ]

1. Pass the chosen id as --folder . folderId accepts a workspace id or a folder id.

If a destination is already supplied, honor it silently — don't ask.

Convert a Looker LookML semantic model into a Sigma data model, then build Sigma workbook(s) that mirror the Looker dashboards (user-defined OR LookML-defined) as closely as possible — and verify the numbers match Looker AND the warehouse.

Read ALL of the following before replying or taking any action. Do not make assumptions about skill conventions, prompts, or global instructions — read the files.

• refs/operating-contract.md — READ FIRST: the fidelity guardrails (render + value-check EVERY page against the source; never ship empty or silently drop a tile; don't spin — surface blockers).
• refs/dashboard-contract.md — the normalized Looker Dashboard JSON contract both the live API fetch and the offline LookML parse produce. The dashboard pipeline is source-agnostic; it only sees this contract.
• refs/looker-dashboard-layout.md — the deep desk study: Looker layout modes, newspaper→24-col grid math, tile-type / filter-type maps, and the full translation-hazard catalog (Liquid, merged_results, table calcs, view/explore field resolution, cross-filtering). This is the design backbone of the dashboard pipeline.
• refs/layered-lookml.md — layered/derived LookML: derived tables on derived tables, cross-view ${view.SQL_TABLE_NAME} refs (CTE inlining vs LOOKER_SCRATCH placeholders), CTE-continuation fragments, incremental/persisted PDTs → the Sigma materialization handoff, dimension_group edge cases, and untranslatable formatting measures. Read before converting any project with derived_table: views.

For canonical spec shape (data-model element kinds, workbook element kinds, controls, formulas, formatting), defer to the companion sigma-data-models and sigma-workbooks skills. This skill restates only the Looker-conversion-specific patterns.

---

The two artifacts, two pipelines

Looker has two independent layers; convert them separately.

| Layer | Source (production = API-first) | Converter | Sigma output | |---|---|---|---| | Semantic model | LookML views+model (Looker API/MCP, or files offline) | local vendored converter/lookml.mjs (run in-process via node; the convert_lookml_to_sigma MCP tool is a manual fallback only) | data model | | Dashboards | GET /dashboards/{id} JSON — covers user-defined (UDD) AND LookML dashboards | fetch_looker_dashboard.py → contract → build_workbook.py | workbook |

Critical — UDD is the primary path. Most real Looker dashboards are user-defined (UDD) — built in the UI, NOT in any LookML file. They are reachable ONLY via the Looker API, which returns UDD and LookML dashboards as the same Dashboard JSON (dashboard_elements[] + dashboard_layouts[] + dashboard_filters[]). So the dashboard converter keys off that API JSON, not LookML. .dashboard.lookml parsing is a secondary, offline-only path that normalizes into the same contract.

---

ONE COMMAND (preferred): migrate-looker.py

The whole pipeline — parse → RLS gate → convert → DM-reuse check → DM POST + readback → workbook build (layout inline) → source-freshness preflight → scripted parity + hard gate — as a single command (mirrors qlik-to-sigma's migrate-qlik.rb / thoughtspot's migrate-thoughtspot.py). Gates are never bypassed: the command exits non-zero if parity or assert-phase6-ran.rb fails.

# env: SIGMA_CONNECTION_ID = the FULL warehouse-connection UUID (NOT a short
# prefix) — required unless --reuse-dm. Persist it once via the tableau plugin's
# `ruby scripts/setup.rb` (writes ~/.sigma-migration/env, which this command
# auto-sources) or export it for the run:
export SIGMA_CONNECTION_ID=
# offline (.dashboard.lookml + view files; the fixture pair works end-to-end):
python3 scripts/migrate-looker.py --lookml-dir fixtures/skilltest-orders \
    --dashboard fixtures/skilltest-orders/skilltest_orders.dashboard.lookml \
    [--name PREFIX] [--workdir /tmp/look-run]
# live (UDD or LookML dashboard, ~/.looker/looker.ini configured):
python3 scripts/migrate-looker.py --lookml-dir /path/to/lookml \
    --dashboard-id  [--explore ] [--name PREFIX] [--workdir DIR]

• Decision points are flags with safe defaults, never silent: detect_rls.py

runs first — RLS findings STOP the command (exit 10, nothing posted) until you either port them via apply_sigma_rls.py (Phase 1.5) or re-run with --yes (proceed WITHOUT RLS — loud + recorded). The DM-reuse check (Phase 2.5) always runs and PRINTS candidates+scores. Default is BUILD-NEW — reuse only when you pin one with --reuse-dm . (Auto-reuse keyed on table coverage could adopt a DM missing a column the workbook needs → the workbook POST then 400s Dependency not found; that footgun is now opt-in via --reuse-auto.) Skip the scan entirely with --skip-dm-reuse-check. The folder is auto-resolved + printed.

• Source repointing: if the LookML sql_table_name points at a DB.SCHEMA the

Sigma connection doesn't serve (e.g. dev CSA.TJ.* vs the connection's QUICKSTARTS.LOOKER_RETAIL_ANALYTICS.*), pass --source-swap FROM_DB.FROM_SCHEMA=TO_DB.TO_SCHEMA (repeatable). A not-yet-indexed schema (catalog miss) self-heals — post_dm.py auto-syncs and retries once. Don't know the FROM? --auto-source-swap-to TO_DB.TO_SCHEMA asks Looker what DB.SCHEMA the explore's connection targets (GET /connections) and builds the swap for you (production-safe; needs ~/.looker/looker.ini).

• No local checkout? --project pulls the LookML over the Looker REST API

(model + views) instead of --lookml-dir. Requires DEVELOP permission on the project (Looker only serves raw LookML in the dev workspace); without it the command fails loud and tells you to clone the Git repo and use --lookml-dir. scripts/looker_project.py is the standalone helper (pull / connection).

• Converter — zero-config, local, no MCP. A self-contained converter bundle

ships in the skill at converter/lookml.mjs and is the default: conversion runs locally via node with no clone, no npm install, no network, no MCP. A dev's own build still wins when set — CONVERTER_SRC (src/lookml.ts via tsx) or CONVERTER_PATH (build/lookml.js), both auto-located. Refresh the vendored bundle with tools/vendor-converters.sh (see converter/PROVENANCE.json). Only if the bundle is missing AND no build is found does the command fall back to the MCP path: it writes /convert-request.json (the exact convert_lookml_to_sigma arguments) and exits 3 — call the tool, save its JSON, re-run with --converted .

• Parity is fully scripted (the Phase-4 gate below): ACTUAL = Sigma CSV

export per chart; EXPECTED = a Looker inline query (live) or a SOURCE-LookML-derived re-aggregation of the master's warehouse rows (offline — measure semantics from the .view.lkml type:, independent of the builder's formulas). Then phase6-parity-looker.rb --finalize + assert-phase6-ran.rb run automatically. Both per-chart fetch sides run in a bounded 4-wide thread pool (measured 24.3s → 5.8s on the 5-chart fixture); set LOOKER_PARITY_WORKERS=1 to serialize on a loaded warehouse (max is clamped to 4 — warehouse-friendly bursts only).

• Exit codes: 0 GREEN · 3 MCP convert request emitted · 10 RLS decision

needed · 2 built but a gate FAILED. --dry-run = no Sigma POSTs.

---

Scripts

| Script | Purpose | |---|---| | scripts/migrate-looker.py | ONE-COMMAND orchestrator (preferred entry) — chains every phase below + the scripted parity hard gate; see the section above. | | scripts/phase6-parity-looker.rb | Phase 4 (parity gate): two-pass orchestrator — PASS 1 reads the workbook spec → parity-plan.json + per-chart fetch instructions; PASS 2 --finalize runs verify-parity.rb and writes the parity-final.json sentinel the hard gate requires. Same contract as quicksight/thoughtspot/tableau. | | scripts/verify-parity.rb | Phase 4: the comparator (strict set-compare with date-bucket canonicalization; --extract-mode tolerance variant). Vendored from the shared converter copy. | | scripts/assert-phase6-ran.rb | HARD GATE (vendored byte-identical across the 5 plugins — keep the md5 in lockstep): parity ran + PASS, no orphan workbooks, no type=error columns, layout applied, layout lint (gate 6), control lint (gate 7 — dead controls / ghost targets / partial same-page reach / control-scope.json coverage; --skip-control-lint escape, exit 9; see refs/control-parity.md). ruby scripts/assert-phase6-ran.rb --workdir --workbook-id must exit 0 before declaring GREEN. | | scripts/probe-controls.rb | Optional Phase-4 flip test — runtime proof that controls actually filter: per control, exports one in-closure element CSV with and without parameters:{controlId: } (must differ) and, with --check-out-of-closure, an out-of-closure element (must NOT differ). Not the mandatory inner loop. Shared, vendored byte-identical. refs/control-parity.md has the design + the MCP-vs-export answer. | | scripts/get-token.sh | Exchange SIGMA_CLIENT_ID/SIGMA_CLIENT_SECRET → SIGMA_API_TOKEN (~1h TTL). eval "$(scripts/get-token.sh)" | | scripts/looker_api.py | Minimal Looker REST API 4.0 client (no SDK). Reads ~/.looker/looker.ini, logs in via client_credentials, exposes L.call(method, path, body). Caches the bearer per process (thread-safe; one login instead of one per call — ~150ms/call saved, measured 2.4x on a 10-call run) and retries once with a fresh login on 401. CLI: python3 looker_api.py whoami / get / raw GET /lookml_models. | | scripts/fetch_looker_dashboard.py | Phase 1 (live): GET /dashboards/{id} → the normalized contract (refs/dashboard-contract.md). Works for UDD AND LookML dashboards. Self-contained (reads ~/.looker/looker.ini). tileType is read from query.vis_config.type (NOT element.type, which is always "vis"); listen from result_maker.filterables; layout from the active layout's components. | | scripts/parse_lookml_dashboard.py | Phase 1 (offline): parse a .dashboard.lookml (YAML) → the SAME contract. Dev/test only; cannot see UDD dashboards. Requires PyYAML. | | scripts/detect_rls.py | Phase 1 (RLS scan): dependency-free regex scan of a LookML dir/file (and/or model JSON) for row-level-security constructs (access_filter, sql_always_where, access_grant, user_attribute). Prints a structured summary + recommended Sigma mapping per finding (or --json). Prints nothing / exits 0 when there's no RLS (zero-overhead). python3 detect_rls.py [--json] | | scripts/apply_sigma_rls.py | Phase 1.5 (apply RLS): scripted, API-driven RLS port. Reuse-first GET /v2/user-attributes (prints a match before creating); --create → POST /v2/user-attributes; --assign (+--member-id,--value) → POST /v2/user-attributes/{id}/users; --field/--element-id → print the verified RLS calc-col + element-filter snippet, --apply --dm-id → PATCH it into the DM element spec. Read-only / plan-only by default — mutates only on an explicit --create/--assign/--apply flag. Reads $SIGMA_BASE_URL/$SIGMA_API_TOKEN like post_dm.py. | | scripts/convert_dm.mjs | Phase 2: run convertLookMLToSigma against a directory of .lkml files for one explore → a Sigma DM spec JSON + …-warnings.json sidecar. A .model.lkml is optional — with none it converts view-only (each view → standalone element; pass the WHOLE directory so cross-view ${view.SQL_TABLE_NAME} refs resolve — see refs/layered-lookml.md). Bypasses the deployed MCP build (see the converter-build gotcha below). Env: LOOKML_DIR, CONVERTER_SRC; args . | | scripts/lookml-dm-signature.py | Phase 2.5: LookML view files → DM-reuse signature ({warehouse_tables, referenced_columns, measures}) for find-or-pick-dm.rb. Pure, no network. | | scripts/find-or-pick-dm.rb | Phase 2.5: scan existing Sigma DMs and recommend reuse (score = 0.7·column + 0.2·table + 0.1·metric overlap; --auto-pick with tie-window safety). Shared vendor-neutral copy (canonical: tableau-to-sigma; needs scripts/lib/sigma_rest.rb). Non-destructive. | | scripts/shape-preflight.rb | Phase 2.5 (reuse gate): before wiring a workbook to a REUSED DM element, mechanically check the three things a spec POST won't: (1) the element is not visibleAsSource:false (hidden → unusable as a source), (2) needed columns resolve on it, (3) every relationship it reaches columns through is 1:1 (non-unique target key → silent fan-out). Visibility/coverage hard-fail (exit 2); fan-out is surfaced with the exact Sigma-MCP uniqueness query per relationship — run it and feed back via --fanout-results (or --ack-fanout). Non-destructive; needs scripts/lib/sigma_rest.rb. migrate-looker.py auto-runs it on --reuse-dm. | | scripts/post_dm.py | Phase 2: POST a DM spec to /v2/dataModels/spec (auto-finds a writable folder, swaps in the full connection UUID). Env: SIGMA_API_TOKEN, SIGMA_BASE_URL, SIGMA_CONNECTION_ID; args `. | | scripts/buildworkbook.py | **Phase 3:** dashboard contract + the explore's view .lkml files → a Sigma /v2/workbooks/spec body (hidden Data page + master table, one element per tile, controls from filters, newspaper→24-col layout XML). Generates locally; does **not** POST. Handles ratio measures, joined-col Field (alias) naming, table calcs, pivot-flatten + warn. Layout: a top control bar (row 0), a full-width strip of **tall** KPI tiles (height ≥ 6 so titles render), then the remaining tiles shifted down. | | scripts/looker-render-dashboard.py | **Phase 4 (visual QA — SOURCE side):** render a LIVE Looker dashboard to PNG via the Looker render API (POST /rendertasks/dashboards/{id}/png → poll GET /rendertasks/{taskid} until success → GET .../results). Pairs with sigma-export-png.py for source-vs-migrated side-by-side. Reuses looker_api.py ~/.looke

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: twells89
• Source: twells89/sigma-migration-skills
• License: MIT

Install and usage instructions live in the source repository linked above.