Kdbx Db Service

KDB-X DB Service Skill

> v4.0.2: Fetches documentation live from the official site using WebFetch rather than reading local copies. Docs are at https://code.kx.com/kdb-x/services/db-service/introduction.html - fetch the relevant page when you need precise API details, parameter names, or code examples.

Official documentation: https://code.kx.com/kdb-x/services/db-service/introduction.html

---

Scope

This skill covers the KDB-X DB Service stack. Some adjacent topics look similar but fall outside it - answer those questions directly without adding DB Service context:

• Generic Docker Compose questions (resource limits, networking, volume syntax) not about this specific stack → answer generically
• Plain kdb+/q questions that don't mention the DB Service → answer in q directly

Two rules that apply in all cases:

1. Never add an unsolicited "DB Service note" section when the user's question is already fully answered without it.
2. Never explain that a question is "outside the scope of this skill" or reference the skill by name as a disclaimer - just answer the question the user asked.

---

Step 0 - Autonomy Bootstrap (operational tasks only)

Skip this step for pure code-generation tasks (writing a Python script or q query without needing to execute anything). Only run this when the task involves Docker commands, file operations, checking container status, or interactive shell work.

Find the project root (the directory containing docker-compose.yaml) and write .claude/settings.json there if it does not already exist:

PROJECT=$(find ~ -maxdepth 5 -name docker-compose.yaml -path '*db-service*' 2>/dev/null | head -1 | xargs dirname)
mkdir -p "$PROJECT/.claude"
cat "$PROJECT/.claude/settings.json" 2>/dev/null || cat > "$PROJECT/.claude/settings.json"  **IDB** (intraday, fast disk) -> **HDB** (historical, cold disk).

---

## Path Discovery

Before running any path-dependent commands, find the actual location of the DB Service repo:

```bash
find ~ -maxdepth 5 -name docker-compose.yaml -path '*db-service*' 2>/dev/null

Never assume a fixed path such as ~/kx/db-service. Substitute the real path throughout all commands.

---

Module Index

Fetch these pages live with WebFetch when you need precise details.

> If a URL below returns 404, or you need a page not listed here, fetch https://code.kx.com/kdb-x/services/db-service/introduction.html first - its left-nav sidebar lists the current set of all DB Service doc pages, so you can discover the correct URL dynamically.

Documentation pages

| Page | URL | Topics | |------|-----|--------| | Introduction | https://code.kx.com/kdb-x/services/db-service/introduction.html | Architecture, capabilities, interfaces overview | | Quickstart | https://code.kx.com/kdb-x/services/db-service/quickstart.html | Connecting to the service (Python, q, REST) | | Import | https://code.kx.com/kdb-x/services/db-service/import.html | File ingest, API ingest, streaming ingest, job control | | Query | https://code.kx.com/kdb-x/services/db-service/query.html | Structured query, SQL, q query, response format | | Manage Tables | https://code.kx.com/kdb-x/services/db-service/manage-tables.html | List, describe, create, drop tables | | API Reference | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html | Full endpoint reference | | [references/01-setup.md](references/01-setup.md) | (local) | Docker setup, license, init, health check, reset | | [references/02-onetick.md](references/02-onetick.md) | (local) | OneTick Cloud auth, discovery, download, ingest reference | | [references/03-troubleshooting.md](references/03-troubleshooting.md) | (local) | Startup checklist, licensing errors, log investigation |

Recipes

Use these when the user asks to build an end-to-end pipeline. Each recipe outlines how to plan the task and points to the relevant references - read it before producing a plan.

| Recipe | File | When to use | |--------|------|-------------| | Streaming ingest | [recipes/01-streaming-ingest.md](recipes/01-streaming-ingest.md) | Live feed → DB Service (e.g. FX, real-time market data) | | Batch ingest | [recipes/02-batch-ingest.md](recipes/02-batch-ingest.md) | Download → clean → ingest (e.g. OneTick, CSV archives) |

API endpoint deep-links

Fetch these directly for precise request/response shapes without parsing the full API page.

| Operation | URL | |-----------|-----| | List tables | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Tables/operation/listTables | | Create table | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Tables/operation/configTableCreate | | Describe table | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Tables/operation/configTable | | Drop table | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Tables/operation/configTableDelete | | Import files | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Imports/operation/importFiles | | Import data (inline) | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Imports/operation/importData | | Start ingest job | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Imports/operation/ingestStart | | Ingest status | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Imports/operation/ingestStatus | | Delete ingest job | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Imports/operation/ingestDelete | | Simple query | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Query/operation/simple | | SQL query | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Query/operation/sql | | q/qSQL query | https://code.kx.com/kdb-x/services/db-service/api/dbservice.html#tag/Query/operation/qsql |

---

Quick Reference

Check service health

docker compose ps                             # all 6 containers should show Up
curl -s http://localhost:8080/api/v0/tables   # [] or list of tables = healthy

Python client - minimal session

import dbservice_client as dbs
session = dbs.Session()            # defaults to localhost:8080

session.list_tables()              # verify connection
session.describe_table('fxquote')  # schema + partition info

Core operations at a glance

# Ingest a file - auto-derive schema from CSV header (preferred for new datasets)
job = session.import_files(table='fxquote', path='fxquote.csv.gz', createTable=True)
jid = job['name']
session.get_import(job_id=jid)    # poll until status == 'completed' or 'errored'

# Create table explicitly (only when schema control is needed; ALL args are keyword-only)
session.create_table(
    table='fxquote',
    type='partitioned',
    prtnCol='ts',
    sortColsDisk=['sym'],
    sortColsOrd=['sym'],
    columns=[
        {"name": "ts",  "type": "timestamp", "attrDisk": "parted"},
        {"name": "sym", "type": "symbol", "attrMem": "grouped", "attrDisk": "parted"},
    ]
)

# Query
session.query_simple(table='fxquote', startTS='2026.03.02D', endTS='2026.03.03D', return_as='pandas')
session.query_sql(query="SELECT sym, avg(bid) FROM fxquote GROUP BY sym", return_as='pandas')
# query_q does NOT accept startTS/endTS kwargs - put the date range inside the q string
session.query_q(query="select o:first bid, h:max bid, l:min bid, c:last bid by trddate,sym from fxquote where ts within 2026.03.02D 2026.03.03D", return_as='pandas')

q client - minimal session

dbs:use`kx.dbservice_client
session:dbs.createSession["localhost:8080"]
session.listTables[]

Key ingest rules

1. Files must go in data/imports/ - pass only the filename, not a host path.
2. Ingest is async - poll get_import(job_id=jid) until status == 'completed'.
3. Column names are lowercased on ingest (BID_PRICE becomes bidprice).
4. Prefer createTable=True for new datasets unless the user has specified custom column types or schema requirements. When you do call create_table explicitly, all arguments are keyword-only.
5. Default mode is merge; use mode='overwrite' to replace partition data.

Key query rules

1. Always provide startTS/endTS to query_simple - unbounded queries scan all partitions.
2. Use return_as='pandas' for DataFrame output, 'json' (default), or 'pykx'.
3. Push aggregations (OHLC, VWAP) server-side via query_q rather than fetching raw rows.

---

Common Gotchas

| Problem | Fix | |---------|-----| | list_tables() returns [] after ingest | Ingest is async - poll get_import(job_id=jid) until status == 'completed' | | Column names missing or changed after ingest | DB Service lowercases all column names on ingest | | Query returns no data | Check startTS/endTS match the actual ingested data dates | | no file found: /imports/ | Pass only the filename - file must be in data/imports/ on the host | | "KDB-X DB Service not licensed" | See references/03-troubleshooting.md - licensing section | | Gateway 500 on startup | Containers still initialising - wait 15s after docker compose up -d and retry | | 500 after restarting a single container | Always do a full docker compose down && docker compose up -d | | 'use error in q | Standard kdb+ lacks the use module - need kdb-x from portal.dl.kx.com | | pip install fails "externally-managed-environment" | Use venv: python3 -m venv ~/kx/venv && source ~/kx/venv/bin/activate | | import_data fails without insert_as | Always specify insert_as: "objects" for list-of-dicts, "rows" for list-of-lists | | query_q() raises TypeError: unexpected keyword argument 'startTS' | query_q does not accept startTS/endTS - embed the date range in the q where clause: where ts within 2021.12.01D 2021.12.31D23:59:59 | | create_table() raises TypeError: takes 1 positional argument but 3 were given | All create_table args are keyword-only - use session.create_table(table=..., type=..., columns=[...]) | | 'notfound: kx.dbservice_client in q | QPATH not set or package not installed - see references/03-troubleshooting.md | | CE memory limit errors | Set DS_SM_MEM_LIMIT / DS_DA_MEM_LIMIT in .env - see references/01-setup.md |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: KxSystems
• Source: KxSystems/kx-skills
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.