BestTester — Gold-Standard QA Platform

[](LICENSE) [](https://nodejs.org) [](https://playwright.dev) [](https://www.typescriptlang.org) [](https://claude.ai) [](https://www.docker.com) [](https://kubernetes.io) [](https://helm.sh)

> The industry-leading, production-grade Playwright + TypeScript QE framework for UI, API, mobile, and AI testing at scale — with distributed Kubernetes execution, Claude vision-based assertions, interactive documentation, comprehensive example gallery, Docker containerization, GitHub Actions optimization, and complete observability via OpenTelemetry.

Why BestTester?

| Advantage | Details | |---|---| | Zero-config start | Clone → npm ci → npm run test:smoke — works out of the box | | Full Page Object Model | Every page extends BasePage; no inline selectors in spec files | | Claude AI Testing | Vision-based assertions, semantic similarity, test generation, hallucination detection | | Interactive Docs | Searchable documentation with getting started, guides, architecture, and 26+ examples | | Example Gallery | Production-ready test patterns for UI, API, security, AI, and mobile testing | | Distributed K8s | Kubernetes Job orchestration with Helm charts, dynamic worker scaling, result aggregation | | Docker Ready | Multi-stage builds, docker-compose stack with Allure + Jaeger, SBOM generation | | CI/CD Optimized | GitHub Actions with npm caching, Playwright browser cache, matrix sharding, change detection | | Observability | OpenTelemetry tracing, Jaeger UI, distributed metrics, structured JSON logging | | Security testing | OWASP Top 10 coverage, XSS/CSRF prevention, auth bypass detection, SQL injection fuzzing | | Multi-environment | TEST_ENV=dev\|staging\|prod with per-environment .env files | | Mobile testing | Dedicated mobile test suite with device emulation (Pixel 5, iPhone 13, iPad), touch/swipe helpers | | Multi-browser | Chromium, Firefox, WebKit, mobile Chrome, mobile Safari via ALL_BROWSERS=true | | Mutation testing | Stryker Mutator with configurable thresholds and HTML/JSON reports | | Data management | Fluent builders, seed management with idempotent application, cleanup registry with audit trails | | Allure + HTML reports | Dual reporting with Allure (GitHub Pages auto-publish) and Playwright HTML | | Code quality | ESLint + Prettier + Husky + Gitleaks secret scanning + pre-commit hooks | | Type safety | Strict TypeScript with Zod schema validation for API responses and requests | | Scaffolding CLI | scaffold:page, scaffold:test, scaffold:api generators for consistent code structure | | File operations | Download verification, Excel read/write, Word document generation, PDF parsing | | i18n testing | Locale switcher, RTL detection, string validation across en, fr, ar, ja | | Auth matrix | Multi-role testing (Admin, Manager, Employee) with cached storage state | | Performance testing | Core Web Vitals measurement, performance budgets, trend tracking, regression detection | | Accessibility testing | WCAG 2.2 AA compliance, axe-core integration, semantic markup validation |

Architecture

┌────────────────────────────────────────────────────────────────────────────┐
│                    BestTester — Gold-Standard QA Platform                  │
├─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬──────────┬────────┤
│UI Tests │API Tests│ Mobile  │AI Tests │Security │  i18n   │ File-Ops │ Perf   │
│(PW)     │(REST)   │(Emulate)│(Claude) │(OWASP)  │(Locale) │(xlsx/pdf)│(CWV)   │
├────────────────────────────────────────────────────────────────────────────┤
│  Page Objects (src/pages/)        │  API Clients (src/api/)         │
│  Mobile Pages (src/mobile/)       │  Security (src/security/)       │
│  AI Integration (src/ai/)         │  Data Builders (src/data/)      │
│  Fixtures (src/fixtures/)         │  i18n (src/i18n/)               │
├────────────────────────────────────────────────────────────────────────────┤
│  AI: Claude Vision │ Semantic Matching │ Test Generation │ Seed Mgr      │
│      Cleanup Registry │ Data Builders (Zod validated)                    │
├────────────────────────────────────────────────────────────────────────────┤
│  Execution: GitHub Actions (Optimized) │ Kubernetes (Distributed)        │
│  Infrastructure: Docker (Multi-stage) │ Helm Charts │ Docker Compose    │
├────────────────────────────────────────────────────────────────────────────┤
│  Reports: Allure │ Playwright HTML │ OTel Traces │ Jaeger │ Metrics      │
│  Observability: OpenTelemetry Tracing │ Winston Structured Logging       │
└────────────────────────────────────────────────────────────────────────────┘

Quick Start

# 1. Clone
git clone https://github.com/your-org/BestTester.git && cd BestTester

# 2. Install
npm ci

# 3. Configure
cp .env.example .env   # edit with your credentials — NEVER commit .env

# 4. Install browsers
npm run setup

# 5. Run smoke tests
npm run test:smoke

Directory Structure

BestTester/
├── .github/workflows/      # CI/CD: ci, smoke, docker-publish, docs deployment
├── config/                 # Playwright configs, global setup, per-environment .env (dev/staging/prod)
├── docs/                   # Interactive documentation (getting started, guides, examples)
│   ├── getting-started/    # quickstart, installation, first-test
│   ├── guides/             # architecture, test-writing, ai-testing, ci-cd, kubernetes
│   └── examples/           # linked to example gallery
├── examples/               # 26+ production-ready test patterns
│   ├── ui/                 # form-validation, file-upload, interactions, responsive
│   ├── api/                # error-handling, authentication, pagination
│   ├── ai/                 # vision-testing, semantic-similarity
│   ├── security/           # xss-prevention, csrf-protection, auth-bypass
│   └── README.md           # example gallery with learning path
├── k8s/                    # Kubernetes configuration
│   ├── job.yaml            # Kubernetes Job manifest
│   ├── configmap.yaml      # Non-secret configuration
│   ├── rbac.yaml           # ServiceAccount, Role, RoleBinding
│   ├── secret.yaml.template # Secrets template (not committed)
│   ├── helm/               # Helm chart for deployment
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   │   └── templates/      # job, configmap, secret, pvc, rbac
│   └── DEPLOYMENT_GUIDE.md # Complete K8s deployment guide
├── scripts/                # Utility scripts
│   ├── k8s-run.ts          # Kubernetes job runner with log streaming
│   ├── create-test.ts      # Interactive test generation CLI
│   └── other scaffolding scripts
├── src/
│   ├── ai/                 # Claude API integration, vision, semantic matching
│   ├── api/                # REST/GraphQL clients with Zod validation
│   ├── auth/               # Multi-role auth, storage state caching
│   ├── data/               # Fluent builders, seed manager, cleanup registry
│   ├── fixtures/           # Custom Playwright fixtures
│   ├── helpers/            # Wait helpers, formatters, utils
│   ├── observability/      # OpenTelemetry tracing, metrics
│   ├── page-objects/       # Page Object Model hierarchy
│   ├── mobile/             # Mobile page objects
│   ├── types/              # TypeScript interfaces
│   └── utils/              # Logger (Winston), file handlers, etc.
├── tests/                  # Comprehensive test suites
│   ├── ui/                 # UI tests (smoke, regression, visual, e2e)
│   ├── api/                # API tests with schema validation
│   ├── ai/                 # AI/LLM tests
│   ├── security/           # Security/OWASP testing
│   ├── mobile/             # Mobile device emulation
│   ├── performance/        # Performance & Core Web Vitals
│   ├── a11y/               # Accessibility (WCAG 2.2 AA)
│   ├── data/               # Data management tests
│   └── flakiness/          # Flaky test detection & quarantine
├── Dockerfile              # Multi-stage build with Playwright base
├── docker-compose.yml      # Full stack: app, testsuite, allure, otel, jaeger
├── otel-collector-config.yml # OpenTelemetry collector configuration
├── package.json            # 50+ npm scripts for all testing scenarios
└── README.md               # This file

Running Tests

# All tests
npm test

# By type
npm run test:ui              # UI tests
npm run test:api             # API tests
npm run test:ai              # AI/LLM tests
npm run test:mobile          # Mobile device emulation tests
npm run test:file-ops        # File operation tests
npm run test:security        # Security tests
npm run test:i18n            # Internationalization tests
npm run test:auth-matrix     # Multi-role auth tests
npm run test:contracts       # Consumer-driven contract tests (Pact)
npm run test:a11y            # Accessibility tests (WCAG 2.2 AA)
npm run test:performance     # Performance & Core Web Vitals tests

# By tag
npm run test:smoke           # @smoke
npm run test:regression      # @regression
npm run test:e2e             # @e2e
npm run test:visual          # @visual
npm run test:a11y            # @a11y
npm run test:performance     # @performance

# Debug modes
npm run test:headed          # Headed browser
npm run test:debug           # Playwright inspector

# Mutation testing
npm run test:mutation        # Stryker mutation analysis

Per Environment

TEST_ENV=dev npm test
TEST_ENV=staging npm test
TEST_ENV=prod npm test

Cross-Browser

ALL_BROWSERS=true npm test   # Chromium + Firefox + WebKit + mobile Chrome + mobile Safari

Mobile Testing

npm run test:mobile                              # Pixel 5 + iPhone 13
npm run test:mobile -- --project mobile-chrome   # Android only
npm run test:mobile -- --project mobile-safari   # iOS only
MOBILE_TABLET=true npm run test:mobile           # Include iPad
npm run test:mobile -- --headed                  # Watch execution

AI-Powered Testing

LLM Client (AWS Bedrock)

Built on AWS Bedrock Converse API with Amazon Nova Pro as the default model. Supports chat, summarization, and embeddings (Titan Embed v2).

AI Assertions

Semantic similarity — cosine similarity on Titan embeddings with configurable threshold
Hallucination detection — keyword-based grounding checks
AI locator fallback — sends DOM snapshot to LLM to resolve broken selectors

LLM-as-Judge

Multi-dimensional evaluation system with 6 built-in rubrics:

| Rubric | Use Case | Pass Threshold | |---|---|---| | Standard | General response quality | 3.5 | | RAG Faithfulness | Grounding in source documents | 4.0 | | Customer Support | Empathy, accuracy, actionability | 3.5 | | Code Generation | Correctness, security, structure | 4.0 | | Summarization | Coverage, conciseness, faithfulness | 3.5 | | Toxicity & Safety | Hate speech, PII, prompt injection | 4.5 |

Scores across 6 dimensions: relevance, accuracy, coherence, safety, faithfulness, instruction adherence.

Anti-narcissistic bias: Primary model (Nova Pro) and judge model (Claude Haiku) use different architectures to prevent self-evaluation bias. Borderline scores (2–3) trigger automatic strict re-evaluation.

Semantic Assertions API

import { semanticExpect } from '@ai/semantic-assert';

await semanticExpect(response).toBeRelevantTo(prompt);
await semanticExpect(response).toBeGroundedIn(sourceContext);
await semanticExpect(response).toBeSafe();
await semanticExpect(response).toNotHallucinate(knownFacts);
await semanticExpect(response).toBeBetterThan(alternativeResponse);
await semanticExpect(response).toHaveScore({ min: 4.0, rubric: RUBRICS.CODE_GENERATION });

MCP Server

Model Context Protocol server for AI-driven test orchestration. Supports both stdio and SSE transports.

npm run mcp:server           # stdio mode
npm run mcp:server:sse       # SSE mode on port 3001
npm run mcp:generate         # AI test generator client

Available MCP tools: | Tool | Description | Example | |---|---|---| | navigate | Open URL, return title + screenshot | navigate({url: "https://example.com"}) | | screenshot | Full-page screenshot of any URL | screenshot({url: "https://example.com"}) | | get_page_text | Extract visible text from a page | get_page_text({url: "https://example.com"}) | | run_test | Execute a specific test file | run_test({testFile: "tests/ui/login.spec.ts"}) | | list_tests | List all available test specs | list_tests({}) | | generate_test_from_description | Generate complete test file from natural language feature description | generate_test_from_description({description: "Test user login flow"}) | | generate_page_object | Analyze a URL and generate a full Page Object Model class | generate_page_object({url: "https://example.com/login"}) | | generate_api_test | Generate API test suite from OpenAPI 3.0 specification | generate_api_test({openapi_spec_url: "https://api.example.com/openapi.json"}) | | analyze_test_coverage | Analyze Page Object Model and identify coverage gaps | analyze_test_coverage({page_object_content: "export class LoginPage { ... }"}) |

Test Generator Examples:

Generate a complete login test:

const result = await client.callTool('generate_test_from_description', {
  description: 'User logs in with email and password, verifies dashboard appears'
});
// Returns TypeScript test file ready to run with npm run test:smoke

Generate a Page Object Model by analyzing a live page:

const pom = await client.callTool('generate_page_object', {
  url: 'https://orangehrm-demo.com/web/index.php/auth/login'
});
// Returns class LoginPage with all selectors and helper methods

Generate API test suite from spec:

const apiTest = await client.callTool('generate_api_test', {
  openapi_spec_url: 'https://api.example.com/v1/openapi.json'
});
// Returns complete test suite validating all endpoints and schemas

Identify test coverage gaps:

const gaps = await client.callTool('analyze_test_coverage', {
  page_object_content: pomClassContent
});
// Returns array of missing test scenarios and accessibility checks

AI Agents

Code Review Agent

npm run agent:review -- --file tests/ui/login.spec.ts
npm run agent:review -- --file tests/ui/login.spec.ts --slack   # post to Slack

Test Healer Agent

Auto-detects broken locators in failed tests and suggests AI-powered fixes on a new git branch.

npm run agent:heal -- --test-result reports/playwright-report/results.json

Suggestion Agent

Analyzes page objects + existing test coverage and generates 5–10 new test case suggestions.

npm run agent:suggest -- --page src/pages/employee.page.ts

Jenkins Trigger Agent

npm run agent:jenkins -- --job playwright-tests --params ENV=staging
npm run agent:jenkins -- --job playwright-tests --build 42
npm run agent:jenkins -- --job playwright-tests --build 42 --logs

Jira Sync Agent

Auto-creates/updates/closes Jira bugs from test results. Attaches JUnit XML and links test executions.

npm run agent:jira -- --build latest
npm run agent:jira -- --junit reports/playwright-report/junit.xml --build 42
npm run agent:jira -- --junit reports/playwright-report/junit.xml --dry-run

Run-and-Report Agent

Full workflow: trigger Jenkins → stream console → download artifacts → sync to Jira → notify Slack.

npm run agent:run-and-report
npm run agent:run-and-report -- --suite smoke --jira-key SCRUM-5
npm run agent:run-and-report -- --suite regression --jira-key SCRUM-5 --dry-run

Slack Bot Agent

npx ts-node agents/slack-bot-agent.ts --report reports/playwright-report/results.json

Contract Testing

Consumer-driven contract testing using

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

Author: nshportun
Source: nshportun/BestTester
License: Apache-2.0

Install and usage instructions live in the source repository linked above.

BestTester

Install

About

BestTester — Gold-Standard QA Platform

Why BestTester?

Architecture

Quick Start

Directory Structure

Running Tests

Per Environment

Cross-Browser

Mobile Testing

AI-Powered Testing

LLM Client (AWS Bedrock)

AI Assertions

LLM-as-Judge

Semantic Assertions API

MCP Server

AI Agents

Code Review Agent

Test Healer Agent

Suggestion Agent

Jenkins Trigger Agent

Jira Sync Agent

Run-and-Report Agent

Slack Bot Agent

Contract Testing

Source & license

Reviews

Write a review

Versions

BestTester

Install

About

BestTester — Gold-Standard QA Platform

Why BestTester?

Architecture

Quick Start

Directory Structure

Running Tests

Per Environment

Cross-Browser

Mobile Testing

AI-Powered Testing

LLM Client (AWS Bedrock)

AI Assertions

LLM-as-Judge

Semantic Assertions API

MCP Server

AI Agents

Code Review Agent

Test Healer Agent

Suggestion Agent

Jenkins Trigger Agent

Jira Sync Agent

Run-and-Report Agent

Slack Bot Agent

Contract Testing

Source & license

Reviews

Write a review

Versions

More in Cloud & Infrastructure