Run Testsprite Backend

Run TestSprite Backend

Use TestSprite as an independent HTTP client against a publicly reachable deployment. Build executable Python tests from repository truth, run them through the TestSprite cloud, inspect immutable run evidence, fix the demonstrated layer, and repeat until a fresh run verifies the intended deployed revision.

This workflow is TypeScript-backend-first. For another stack, inspect and substitute that repository's route, schema, test, build, and deployment conventions; TestSprite still exercises the service over HTTP.

Value boundary

TestSprite is valuable here because it behaves like an independent deployed client: it can catch defects that native tests and same-process mocks miss, retain the exact request/response path, and challenge semantic contracts such as citations, streaming order, routing metadata, and typed failures.

It is not a deployment system, account/proxy provider, CAPTCHA solver, load tester, or autonomous repository maintainer. Its AI analysis proposes a cause and fix target; it does not prove either. A non-passing run can still be useful evidence when it correctly shows old production code, unavailable capacity, or an upstream gate.

| TestSprite can prove | It cannot prove by itself | |---|---| | What the public target returned to its HTTP client | That the target serves the commit in the checkout | | Whether saved assertions held for one pinned run | That a suggested root cause is correct | | Request, response, Data Flow, dependency, and artifact evidence | That an account, proxy, provider, or human challenge is healthy | | A fresh external pass after deployment | Native unit correctness, load tolerance, or security completeness |

Essential rules

1. Read the nearest repository instructions before touching code or TestSprite state.
2. Test a public deployment and prove which revision it serves. A green run against old code is not release proof.
3. Store API credentials in TestSprite. Never put secrets in saved test code, command arguments, logs, artifacts, or git.
4. Call every backend test_* function. TestSprite executes Python top-to-bottom; it does not rely on pytest discovery.
5. Use only Python stdlib plus the packages supported by the current backend sandbox. Exercise project code through HTTP, never by importing it.
6. Keep live tests bounded, deterministic, and reversible. Do not turn a verification suite into load, abuse, or destructive production testing.
7. Treat TestSprite's LLM analysis as a hypothesis and saved tests as durable contracts: validate suggestions, then refine and reuse rather than regenerate by default.

When to use this skill

Use it when the user asks to:

• bootstrap or deepen TestSprite backend/API coverage;
• turn OpenAPI and product rules into executable cloud tests;
• debug a failed TestSprite backend run or artifact;
• verify streaming, sources, headers, routing metadata, auth, or error contracts;
• rerun tests after a fix and prove the exact deployed revision; or
• add TestSprite backend verification to CI.

Do not use it for frontend browser plans, local unit tests alone, generic API-client implementation, load/security scanning, or a service with no publicly reachable test target. Use a frontend TestSprite workflow for browser journeys and the repository's native test framework for local tests.

Before spending cloud runs, apply a value test: the scenario must protect a material consumer-visible contract, exercise a deployed boundary unavailable to native tests, or preserve evidence for a meaningful external integration. A duplicate 200-only check has little value.

Reference router

Load only the references needed for the current phase. Every reference is self-contained enough to enter at that phase.

| Situation | Read | |---|---| | Unfamiliar repo, missing project mapping, or uncertain deployed revision | [references/repo-discovery.md](references/repo-discovery.md) | | New suite, coverage gaps, dependencies, streaming, or production-safety design | [references/suite-design.md](references/suite-design.md) | | Writing or auditing TestSprite Python | [references/backend-test-authoring.md](references/backend-test-authoring.md) | | Exact 0.3.0 commands, flags, outputs, exit codes, or official sources | [references/cli-reference.md](references/cli-reference.md) | | Failed, blocked, flaky, deferred, or contradictory results | [references/failure-loop.md](references/failure-loop.md) | | Credentials, CI, artifacts, deployment proof, and final release gate | [references/release-security.md](references/release-security.md) |

Common reading sets:

• First suite: repo discovery + suite design + authoring + release/security.
• Existing-test verification: repo discovery + CLI reference + failure loop.
• Authenticated CI: CLI reference + release/security.
• Streaming/provider failure: authoring + failure loop + release/security.

Workflow

1. Frame a concrete verification goal

Write down:

• behavior or contract to prove;
• public target environment;
• expected deployed revision;
• allowed mutations and cleanup path;
• relevant endpoints and authentication mode; and
• completion condition: a fresh TestSprite run, terminal verdict, and revision evidence.

Separate code correctness from environment availability. An account shortage, CAPTCHA, proxy failure, or upstream outage is not evidence that repository code is wrong; it is also not a pass.

Predeclare the possible end states: verified, product defect, test defect, deployment drift, runtime/provider gate, or TestSprite execution failure. Do not rename an inconvenient result after the run.

2. Discover repository truth

Read repository instructions, package manifests, routes/controllers, OpenAPI or other schemas, product docs, native tests, live harnesses, deployment workflow, and version/health endpoints. For TypeScript, start with package.json, workspace config, route registrations, handlers, contract types, and *.test.ts. For another stack, inspect the equivalent surfaces.

Rank conflicting evidence:

1. observed behavior of the intended deployed revision;
2. executable server code and tests;
3. published API schema;
4. maintained product documentation;
5. old examples and generated prose.

Do not silently normalize contradictions. Record whether the implementation, schema, or deployment is stale. Follow [repo discovery](references/repo-discovery.md) for the full evidence brief.

3. Preflight the current CLI and account

Examples below were verified with TestSprite CLI 0.3.0. Put global flags before the subcommand and let the installed --help win when a newer version differs.

testsprite --version
testsprite --output json auth status
testsprite --output json doctor
testsprite --output json usage

Require 0.3.0 or newer for managed backend credentials. If the CLI is absent, follow the official installation path. If authentication fails, use testsprite setup; never work around it by embedding an API key in test code. doctor can expose connectivity, profile, scope, or stale agent-guidance failures; read its JSON rather than ignoring a nonzero exit.

4. Resolve the backend project and current suite

Resolve in this order:

1. an explicit project ID supplied for the task;
2. TESTSPRITE_PROJECT_ID;
3. a repository .testsprite/config.json with projectId; then
4. project-name matching from the API.

testsprite --output json project list --max-items 100
testsprite --output json project get "$PROJECT_ID"
testsprite --output json test list --project "$PROJECT_ID" --type backend --max-items 100
testsprite --output json test list --project "$PROJECT_ID" --type backend --status failed,blocked

Do not guess when several projects plausibly match. Compare project type, existing test names, repository docs, and recent run targets. A project is one type; create a backend project only if no correct one exists and the task authorizes external project creation.

5. Prove the target before spending a run

TestSprite calls a deployed URL; it does not deploy or host the application. Confirm that the target is public HTTP(S), healthy, and serving the intended revision using the repository's version endpoint, image label, deployment API, or exact-SHA CI evidence.

In TestSprite 0.3.0, backend tests define their base URL inside saved Python. Backend create/run flows may accept the generic --target-url flag, but code-file creation reports that it does not retarget backend code and batch run rejects the override. Verify the saved code and completed run Data Flow point to the intended environment. To change environments, update the saved code optimistically or maintain clearly named environment-specific tests. If the revision cannot be proven, the run may diagnose the current deployment but cannot verify the change.

Capture a target fingerprint before and after the run: base URL, revision or image digest, environment, relevant lane/tenant, and test code version. If production still serves an older revision, a failure reproducing the old bug is correct evidence; deploy first instead of editing the assertion or repeatedly rerunning.

6. Configure authentication without exposing it

Use a secret file populated through an authorized secret manager or protected local path. Do not use --credential in agent-driven workflows because arguments can leak through shell history and process listings.

testsprite --output json project credential "$PROJECT_ID" \
  --type "Bearer token" --credential-file "$CREDENTIAL_FILE"

testsprite --output json project credential "$PROJECT_ID" --type public

Use project auto-auth with its *-file flags for recurring tokens when the plan supports it. In Python, copy __AUTH_HEADERS__ into each authenticated request; never print __AUTH_CREDENTIAL__. Read [release and security](references/release-security.md) before changing credentials or handling old tests that once contained secrets.

7. Design the suite before creating tests

Build a traceable matrix from documented behavior, not a pile of happy-path status checks. Cover, where applicable:

• health/version and one representative success per public capability;
• authentication and authorization failures;
• input validation and typed error bodies;
• response structure plus semantic invariants;
• headers, source URLs, routing or correlation metadata;
• streaming event order, terminal event, and accumulated result;
• idempotency or state transitions;
• external-provider failure typing; and
• cleanup for created fixtures.

Use stable assertions: types, allowed enums, required non-empty values, URL validity, event ordering, and relationships between fields. Avoid exact model prose, timestamps, generated IDs, latency, or provider wording unless the contract guarantees them. See [suite design](references/suite-design.md).

For each scenario, name the independent signal it adds over native tests. Preserve separate non-streaming and streaming cases when the application has separate accumulation/mapping paths; one passing path does not validate the other.

8. Author and audit executable Python

Start from the current local scaffold:

testsprite test scaffold --type backend --out /tmp/testsprite-health.py

Replace its placeholder URL and assertion. A minimal authenticated test looks like this:

import requests

BASE_URL = "https://api.example.com"
AUTH_HEADERS = dict(__AUTH_HEADERS__)

def test_profile_contract() -> None:
    response = requests.get(
        f"{BASE_URL}/v1/profile",
        headers=AUTH_HEADERS,
        timeout=(10, 60),
    )
    assert response.status_code == 200, f"expected 200, got {response.status_code}"
    body = response.json()
    assert isinstance(body.get("id"), str) and body["id"]
    assert body.get("status") in {"active", "limited"}

test_profile_contract()

Replace the example URL with the real public target. __AUTH_HEADERS__ is supplied by the configured TestSprite credential. Never leave example.com in created code, and do not expect --target-url to rewrite backend Python.

Resolve TESTSPRITE_SKILL_DIR to the directory containing this loaded SKILL.md, then run the bundled static auditor before upload. Do not assume the skill is installed inside the target repository.

python3 "$TESTSPRITE_SKILL_DIR/scripts/audit_backend_test.py" \
  --auth-required /tmp/testsprite-profile.py

It rejects syntax errors, dead/uncalled or request-free tests, unsupported imports, missing request timeouts (including Session calls), private/placeholder targets, and likely embedded credentials or cookies without printing secret values. Read [backend authoring](references/backend-test-authoring.md) for JSON, SSE, URL, error, and stateful examples.

9. Preview shape, then create and run one narrow test

--dry-run validates command shape only. It does not execute Python, authenticate to the application, validate the deployed target, or consume a real run.

testsprite --dry-run --output json test create \
  --type backend --project "$PROJECT_ID" --name "profile contract" \
  --code-file /tmp/testsprite-profile.py --run --wait --timeout 600

testsprite --output json test create \
  --type backend --project "$PROJECT_ID" --name "profile contract" \
  --code-file /tmp/testsprite-profile.py --run --wait --timeout 600

Capture the returned project ID, test ID, run ID, target, status, dashboardUrl, and exit code. Exit 7 means the same run is still available: resume it with test wait; do not trigger a duplicate.

10. Diagnose failures from pinned evidence

For a failed, blocked, or cancelled run, download its immutable run-scoped bundle:

mkdir -p .testsprite/runs
testsprite --output json test artifact get "$RUN_ID" \
  --out ".testsprite/runs/$RUN_ID"
testsprite --output json test steps "$TEST_ID" --run-id "$RUN_ID" --max-items 100
testsprite --output json test result "$TEST_ID" --include-analysis

Classify the failure before editing anything: product code, saved test code, stale deployment, application configuration, auth/account capacity, edge/transport, provider/proxy/CAPTCHA, TestSprite runner, dependency starvation, or nondeterminism. Verify the LLM hypothesis against request/response evidence and the repository path. Follow [the failure loop](references/failure-loop.md).

11. Fix the demonstrated layer and preserve the contract

• Product defect: add a native regression, fix the code, pass exact-SHA CI, deploy that SHA, and rerun.
• Test defect: edit only the incorrect setup/parser/assertion; do not weaken the product contract.
• Deployment defect: correct the target/config/revision, prove the new artifact is live, and only then rerun.
• Auth/account/proxy/provider gate: record the exact operational evidence and retry only after the resource state changed.
• Edge timeout: identify the terminating layer; increasing the TestSprite polling timeout does not extend a CDN or gateway request deadline.
• TestSprite execution defect: preserve the run/request IDs and bounded diagnostics; do not edit product code without a faithful application response.
• Dependency starvation: fix the failed producer before judging consumers.

Update saved code with optimistic concurrency:

testsprite test code get "$TEST_ID" --out /tmp/testsprite-current.py
testsprite --output json test code get "$TEST_ID"
testsprite --output json test code put "$TEST_ID" \
  --code-file /tmp/testsprite-fixed.py --expected-version "$CODE_VERSION"

Do not use --force unless you have proved the competing edit is disposable.

12. Expand safely, then finish with a fresh run

Declare real producer/consumer/teardown relationships at create time with --produces, --needs, and --category teardown. Change that graph by deleting and recreating the affected tests; current read/update commands do not round-trip the graph.

Use test run --all only when

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: yigitkonur
• Source: yigitkonur/skills-by-yigitkonur
• License: MIT

Install and usage instructions live in the source repository linked above.