Cortex

Cortex

[](https://www.npmjs.com/package/@aj-archipelago/cortex) [](LICENSE) [](package.json)

Cortex is an open-source AI backend control plane: model routing, agent tools, OpenAI-compatible APIs, and private workspaces behind one programmable runtime.

It is built for teams that want the modern AI stack without welding their product to one provider SDK, one brittle prompt chain, or one proprietary agent loop.

What You Get

• One API for many providers. Route OpenAI, Azure OpenAI, Gemini, Claude on Vertex, Grok, Replicate-hosted media models, Ollama, local models, and custom provider plugins through GraphQL, REST, OpenAI-compatible chat/completions/responses, and Anthropic-style messages APIs.
• Routing that can adapt. Use model groups, redirects, per-request model overrides, endpoint health, duplicate-request hedging, and background latency sampling so "the default model" can be a strategy instead of a hardcoded string.
• An agent harness you can own. sys_entity_agent combines entity configuration, tools, MCP discovery, client-side tools, request-scoped tools, streaming progress, tool-result compaction, and memory-aware context into one reusable agent pathway.
• Private workspaces for real work. Attach each entity to an isolated Docker or Azure Container Instances workspace with shell access, file APIs, checkpoint/restore, warm-pool provisioning, and secret injection.
• Simple extension points. Add a capability with one pathway file, then graduate to executePathway when you need validation, orchestration, custom tools, provider-specific handling, or richer result metadata.

Why Cortex Exists

The frontier AI product pattern is getting clearer: multi-model routing, agentic tool loops, entity personalization, memory, MCP-style tool discovery, client-side tool callbacks, and dedicated agent workspaces. Big products and funded startups are converging on those pieces because serious AI apps need more than a chat wrapper.

Cortex puts those pieces in one open backend that you can run, inspect, customize, and embed.

Model APIs keep changing. Capabilities move between providers. Latency shifts by region and hour. Agent tool catalogs grow until the model drowns in schemas. Workspace execution needs isolation, persistence, and recoverability. Product teams still need one stable API.

Cortex turns that mess into infrastructure:

• A model catalog with provider-specific execution plugins.
• A model router that can redirect old model ids, expose model aliases, and pick healthy group members using latency samples.
• A GraphQL schema generated from pathways, entities, and dynamic configuration.
• Optional REST surfaces for generic pathways and provider-compatible clients.
• An agent harness that can discover tools only when needed, execute them, stream progress, compact results, and continue the reasoning loop.
• A workspace layer that can provision private sandboxes locally or in Azure and restore durable state after idle reaping.

Why Cortex Instead Of...

| Alternative | Good at | Where Cortex is different | | --------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | | Provider SDKs | Direct access to one provider's newest API | Cortex keeps product code behind a stable runtime while providers, models, and protocols change. | | Model proxies | Unifying model calls and keys | Cortex also has pathways, entities, tools, streaming progress, memory-aware agents, and workspaces. | | Prompt-chain frameworks | Fast experimentation inside an app | Cortex is a backend service with GraphQL, REST, auth, routing, cancellation, caching, and operational policy. | | Proprietary agent platforms | Polished hosted loops | Cortex gives you the loop, tool surface, and workspace architecture in code you can own. |

Start Here In 5 Minutes

If you are new to Cortex, pick the lane closest to what you are building:

| If you want to... | Start with... | Why | | --------------------------------------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Put one stable API in front of many model providers | [Model Configuration](#model-configuration) and [REST](#rest) | Define models once, expose GraphQL or OpenAI-compatible REST, and move callers with redirects/groups later. | | Build an agent with tools | [Agents And Entities](#agents-and-entities), then [Pathways](#pathways) | Entities define identity and tool access; pathways define the callable skills behind that agent. | | Give agents private compute | [Workspace Architecture](#workspace-architecture) | Workspaces give each entity a container for shell commands, files, checkpoints, and long-running work. | | Add a new capability | [Pathways](#pathways) | Most Cortex extensions are one pathway file plus, optionally, executePathway for orchestration. |

The shortest path is:

1. Run Cortex locally.
2. Call a built-in pathway through GraphQL.
3. Enable OpenAI-compatible REST and call a model or agent.
4. Add one custom pathway.
5. Turn on entities, tools, and workspaces when your product needs them.

You do not need to understand every provider plugin or workspace knob before Cortex is useful.

Try It

git clone git@github.com:aj-archipelago/cortex.git
cd cortex
npm install
CORTEX_ENABLE_REST=true npm start

Set OPENAI_API_KEY in your shell, .env, or .env.local before starting. Cortex loads .env and .env.local automatically for local runs; explicit shell variables win.

By default Cortex starts GraphQL at http://localhost:4000/graphql. From another terminal:

curl http://localhost:4000/healthcheck

Then call a pathway through Cortex's generated REST API:

curl http://localhost:4000/rest/summary \
  -H 'content-type: application/json' \
  -d '{
    "text": "Cortex routes model requests, runs pathways, and powers agentic tools."
  }'

Or call the same pathway through Cortex's generated GraphQL schema:

curl http://localhost:4000/graphql \
  -H 'content-type: application/json' \
  -d '{
    "query": "query($text: String!) { summary(text: $text) { result } }",
    "variables": {
      "text": "Cortex routes model requests, runs pathways, and powers agentic tools."
    }
  }'

What Cortex Is Not

Cortex is not a UI framework, a prompt collection, or a thin SDK wrapper. It is the backend layer you put between AI-facing product code and the parts that keep changing: providers, model ids, tool schemas, streaming formats, workspace execution, and operational policy.

That means Cortex is best when you want:

• one internal AI API instead of provider SDKs scattered through your app,
• model upgrades without client rewrites,
• agents that can discover and use real tools,
• private compute for agent work,
• enough structure to operate this in production.

Core Concepts

1. Open Model Router

Cortex model configs describe the provider, endpoint, credentials, request shape, metadata, REST emulation name, and media controls for each model. The router then handles:

• Provider plugins for OpenAI, OpenAI Responses, Azure OpenAI, Gemini, Claude/Anthropic, Grok/xAI, Replicate, VEO, Ollama, local models, embeddings, transcription, TTS, music, image, and video models.
• modelRedirects for moving callers off old ids without touching every client.
• modelGroups for aliases such as "default coding model" or "agent chat model" whose members can be selected dynamically.
• Background latency sampling for model-group members that have not seen recent comparable traffic.
• Endpoint health and fastest-endpoint selection inside each model.
• Request-level model override through pathway args.
• OpenAI-compatible REST exposure through emulateOpenAIChatModel and emulateOpenAICompletionModel.

The point is simple: product code should ask for the capability it wants. Cortex decides where that request should land.

2. Entity Agent Harness

sys_entity_agent is the main agentic runtime. It is a pathway, but it behaves like an agent harness:

• Loads an entity configuration by id or default entity.
• Resolves entity tools from global system tools plus entity-specific custom tools.
• Supports lazy tool discovery through SearchAvailableTools so the model does not need every schema upfront.
• Discovers MCP tools and can hot-load matched tools into the live request.
• Accepts caller-provided client-side tools and waits for client results.
• Supports request-scoped tools such as reauthentication and tool-result inspection.
• Runs the tool loop, sends structured progress events, enforces tool budgets, detects duplicate calls, compacts large results, and re-enters the model after each tool batch.
• Accepts pending user-message injection during long running streams.
• Can work through GraphQL or through the OpenAI-compatible /v1/chat/completions surface as model: "cortex-agent" when REST endpoints are enabled.

Entities are where you define personality, instructions, tool access, memory behavior, workspace behavior, and required environment. The harness is where that configuration becomes an actual runtime.

3. Private Containerized Workspaces

Cortex workspaces are isolated execution environments owned by entities. They are designed for agentic coding, research, file processing, data work, and other tasks where the agent should compute instead of bluffing.

The workspace stack includes:

• WorkspaceSSH, a consolidated shell tool with foreground commands, background jobs, polling, reset, restore, and destroy flows.
• helper-apps/cortex-workspace, a lightweight HTTP helper running inside the sandbox.
• Docker backend for local development.
• Azure Container Instances backend for hosted private workspaces.
• Warm pool support for faster first command latency.
• Per-workspace secret headers and secret rotation when warm containers are claimed.
• File upload, download, read, write, edit, browse, shell, status, backup, restore, and reset endpoints.
• Blob-backed checkpoint and restore for ACI workspaces.
• Optional encrypted checkpoints with ownership metadata validation.
• Idle checkpointing and idle reaping so sleeping workspaces stop burning compute while preserving useful state.

This follows the same pattern that has emerged in OpenClaw/NanoClaw-style systems: each agent gets a private, containerized machine room, not a shared scratchpad pretending to be isolation.

Install As A Package

npm install @aj-archipelago/cortex

import cortex from '@aj-archipelago/cortex';

const { startServer } = await cortex({
  PORT: 4000,
  defaultModelName: 'oai-gpt54-mini',
});

await startServer();

API Surfaces

GraphQL

Every enabled pathway becomes a GraphQL field. In development, the embedded Apollo landing page is available at /graphql.

Example:

query Translate($text: String!, $to: String!) {
  translate(text: $text, to: $to) {
    result
  }
}

Agent example:

query Agent($text: String!, $entityId: String) {
  sys_entity_agent(text: $text, entityId: $entityId, stream: false) {
    result
    resultData
  }
}

GraphQL also exposes:

• requestProgress subscriptions for streaming/progress events.
• cancelRequest mutation for cancellation.
• submitClientToolResult mutation for client-side tool callbacks.
• injectAgentMessage mutation for long-running agent loops.
• executeWorkspace query for controlled workspace execution.

REST

REST is off by default. Enable it with:

export CORTEX_ENABLE_REST=true

When enabled, Cortex registers:

• POST /rest/{pathwayName} for non-emulation pathways.
• POST /v1/chat/completions for OpenAI-compatible chat models.
• POST /v1/completions for OpenAI-compatible completion models.
• POST /v1/responses for OpenAI Responses-style clients.
• POST /v1/messages for Anthropic-style clients.
• GET /v1/models for exposed REST models.

OpenAI-compatible agent call:

curl http://localhost:4000/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "cortex-agent",
    "messages": [
      { "role": "user", "content": "Create a short launch checklist for Cortex." }
    ],
    "stream": true
  }'

OpenAI-compatible model call:

curl http://localhost:4000/v1/responses \
  -H 'content-type: application/json' \
  -d '{
    "model": "gpt-5.4-mini",
    "input": "Explain model groups in one paragraph."
  }'

If CORTEX_API_KEY is set, Cortex accepts the key through cortex-api-key, Authorization: Bearer ..., or x-api-key.

Model Configuration

Model configuration lives in config/default.example.json and can be overridden through CORTEX_CONFIG_FILE, direct config passed to the package, or environment variables.

Minimal custom model shape:

{
  "models": {
    "my-openai-model": {
      "name": "my-openai-model",
      "type": "OPENAI-RESPONSES",
      "supportsStreaming": true,
      "emulateOpenAIChatModel": "my-model",
      "endpoints": [
        {
          "name": "openai",
          "url": "https://api.openai.com/v1/responses",
          "headers": {
            "Authorization": "Bearer {{OPENAI_API_KEY}}"
          },
          "params": {
            "model": "gpt-5.4-mini"
          }
        }
      ],
      "metadata": {
        "displayName": "My Model",
        "provider": "openai",
        "category": "chat"
      }
    }
  }
}

Model Redirects

Use redirects to move clients forward without breaking old callers:

{
  "modelRedirects": {
    "old-default": "oai-gpt54-mini",
    "gpt-4o": "oai-gpt54-mini"
  }
}

Redirects resolve before endpoint lookup and before model-group selection.

Model Groups

Use model groups when a "model" should really be a ranked capability pool:

{
  "modelGroups": {
    "cortex-agent-chat": {
      "members": [
        "oai-gpt54-mini",
        "gemini-flash-35-vision",
        "claude-46-sonnet-vertex"
      ],
      "metadata": {
        "displayName": "Fast Agent Chat",
        "provider": "cortex",
        "category": "chat",
        "isAgentic": true
      }
    }
  }
}

The picker uses sampler-owned ping TTFB so group members are compared against the same kind of request. Members within the slowness tolerance of the fastest sample are selected by priority order. If no usable latency data exists yet, Cortex falls back to the first configured member.

Current Built-In Model Families

The open-source catalog is intentionally current-leaning. It includes modern chat, reasoning, vision, image, video, music, speech, embeddings, transcription, and hosted media models across:

• OpenAI Responses and image/audio models.
• Azure OpenAI deployments.
• Gemini 3.x chat, reasoning, image, TTS, and music models.
• Claude 4.x on Vertex.
• Grok/xAI chat, reasoning, responses, and code models.
• VEO 3.1 video variants.
• Replicate-hosted image, video, and speech models.
• Ollama and local model adapters.

Clients can inspect publishable model metadata through sys_model_metadata, including display names, provider, category, group status, media controls, requir

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: aj-archipelago
• Source: aj-archipelago/cortex
• License: MIT
• Homepage: https://www.npmjs.com/package/@aj-archipelago/cortex

Install and usage instructions live in the source repository linked above.