# BestTester

> Production-grade Playwright + TypeScript QA framework with AI-powered testing, LLM-as-Judge evaluation, MCP server, 7 CLI agents, security fuzzing, CI/CD pipelines, Jira sync, and Slack reporting — zero-config, plug-and-play.

- **Type:** MCP server
- **Install:** `agentstack add mcp-nshportun-besttester`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [nshportun](https://agentstack.voostack.com/s/nshportun)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [nshportun](https://github.com/nshportun)
- **Source:** https://github.com/nshportun/BestTester

## Install

```sh
agentstack add mcp-nshportun-besttester
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# BestTester — Gold-Standard QA Platform

[](LICENSE)
[](https://nodejs.org)
[](https://playwright.dev)
[](https://www.typescriptlang.org)
[](https://claude.ai)
[](https://www.docker.com)
[](https://kubernetes.io)
[](https://helm.sh)

> **The industry-leading, production-grade Playwright + TypeScript QE framework** for UI, API, mobile, and AI testing at scale — with distributed Kubernetes execution, Claude vision-based assertions, interactive documentation, comprehensive example gallery, Docker containerization, GitHub Actions optimization, and complete observability via OpenTelemetry.

---

## Why BestTester?

| Advantage | Details |
|---|---|
| **Zero-config start** | Clone → `npm ci` → `npm run test:smoke` — works out of the box |
| **Full Page Object Model** | Every page extends `BasePage`; no inline selectors in spec files |
| **Claude AI Testing** | Vision-based assertions, semantic similarity, test generation, hallucination detection |
| **Interactive Docs** | Searchable documentation with getting started, guides, architecture, and 26+ examples |
| **Example Gallery** | Production-ready test patterns for UI, API, security, AI, and mobile testing |
| **Distributed K8s** | Kubernetes Job orchestration with Helm charts, dynamic worker scaling, result aggregation |
| **Docker Ready** | Multi-stage builds, docker-compose stack with Allure + Jaeger, SBOM generation |
| **CI/CD Optimized** | GitHub Actions with npm caching, Playwright browser cache, matrix sharding, change detection |
| **Observability** | OpenTelemetry tracing, Jaeger UI, distributed metrics, structured JSON logging |
| **Security testing** | OWASP Top 10 coverage, XSS/CSRF prevention, auth bypass detection, SQL injection fuzzing |
| **Multi-environment** | `TEST_ENV=dev\|staging\|prod` with per-environment `.env` files |
| **Mobile testing** | Dedicated mobile test suite with device emulation (Pixel 5, iPhone 13, iPad), touch/swipe helpers |
| **Multi-browser** | Chromium, Firefox, WebKit, mobile Chrome, mobile Safari via `ALL_BROWSERS=true` |
| **Mutation testing** | Stryker Mutator with configurable thresholds and HTML/JSON reports |
| **Data management** | Fluent builders, seed management with idempotent application, cleanup registry with audit trails |
| **Allure + HTML reports** | Dual reporting with Allure (GitHub Pages auto-publish) and Playwright HTML |
| **Code quality** | ESLint + Prettier + Husky + Gitleaks secret scanning + pre-commit hooks |
| **Type safety** | Strict TypeScript with Zod schema validation for API responses and requests |
| **Scaffolding CLI** | `scaffold:page`, `scaffold:test`, `scaffold:api` generators for consistent code structure |
| **File operations** | Download verification, Excel read/write, Word document generation, PDF parsing |
| **i18n testing** | Locale switcher, RTL detection, string validation across `en`, `fr`, `ar`, `ja` |
| **Auth matrix** | Multi-role testing (Admin, Manager, Employee) with cached storage state |
| **Performance testing** | Core Web Vitals measurement, performance budgets, trend tracking, regression detection |
| **Accessibility testing** | WCAG 2.2 AA compliance, axe-core integration, semantic markup validation |

---

## Architecture

```
┌────────────────────────────────────────────────────────────────────────────┐
│                    BestTester — Gold-Standard QA Platform                  │
├─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬──────────┬────────┤
│UI Tests │API Tests│ Mobile  │AI Tests │Security │  i18n   │ File-Ops │ Perf   │
│(PW)     │(REST)   │(Emulate)│(Claude) │(OWASP)  │(Locale) │(xlsx/pdf)│(CWV)   │
├────────────────────────────────────────────────────────────────────────────┤
│  Page Objects (src/pages/)        │  API Clients (src/api/)         │
│  Mobile Pages (src/mobile/)       │  Security (src/security/)       │
│  AI Integration (src/ai/)         │  Data Builders (src/data/)      │
│  Fixtures (src/fixtures/)         │  i18n (src/i18n/)               │
├────────────────────────────────────────────────────────────────────────────┤
│  AI: Claude Vision │ Semantic Matching │ Test Generation │ Seed Mgr      │
│      Cleanup Registry │ Data Builders (Zod validated)                    │
├────────────────────────────────────────────────────────────────────────────┤
│  Execution: GitHub Actions (Optimized) │ Kubernetes (Distributed)        │
│  Infrastructure: Docker (Multi-stage) │ Helm Charts │ Docker Compose    │
├────────────────────────────────────────────────────────────────────────────┤
│  Reports: Allure │ Playwright HTML │ OTel Traces │ Jaeger │ Metrics      │
│  Observability: OpenTelemetry Tracing │ Winston Structured Logging       │
└────────────────────────────────────────────────────────────────────────────┘
```

---

## Quick Start

```bash
# 1. Clone
git clone https://github.com/your-org/BestTester.git && cd BestTester

# 2. Install
npm ci

# 3. Configure
cp .env.example .env   # edit with your credentials — NEVER commit .env

# 4. Install browsers
npm run setup

# 5. Run smoke tests
npm run test:smoke
```

---

## Directory Structure

```
BestTester/
├── .github/workflows/      # CI/CD: ci, smoke, docker-publish, docs deployment
├── config/                 # Playwright configs, global setup, per-environment .env (dev/staging/prod)
├── docs/                   # Interactive documentation (getting started, guides, examples)
│   ├── getting-started/    # quickstart, installation, first-test
│   ├── guides/             # architecture, test-writing, ai-testing, ci-cd, kubernetes
│   └── examples/           # linked to example gallery
├── examples/               # 26+ production-ready test patterns
│   ├── ui/                 # form-validation, file-upload, interactions, responsive
│   ├── api/                # error-handling, authentication, pagination
│   ├── ai/                 # vision-testing, semantic-similarity
│   ├── security/           # xss-prevention, csrf-protection, auth-bypass
│   └── README.md           # example gallery with learning path
├── k8s/                    # Kubernetes configuration
│   ├── job.yaml            # Kubernetes Job manifest
│   ├── configmap.yaml      # Non-secret configuration
│   ├── rbac.yaml           # ServiceAccount, Role, RoleBinding
│   ├── secret.yaml.template # Secrets template (not committed)
│   ├── helm/               # Helm chart for deployment
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   │   └── templates/      # job, configmap, secret, pvc, rbac
│   └── DEPLOYMENT_GUIDE.md # Complete K8s deployment guide
├── scripts/                # Utility scripts
│   ├── k8s-run.ts          # Kubernetes job runner with log streaming
│   ├── create-test.ts      # Interactive test generation CLI
│   └── other scaffolding scripts
├── src/
│   ├── ai/                 # Claude API integration, vision, semantic matching
│   ├── api/                # REST/GraphQL clients with Zod validation
│   ├── auth/               # Multi-role auth, storage state caching
│   ├── data/               # Fluent builders, seed manager, cleanup registry
│   ├── fixtures/           # Custom Playwright fixtures
│   ├── helpers/            # Wait helpers, formatters, utils
│   ├── observability/      # OpenTelemetry tracing, metrics
│   ├── page-objects/       # Page Object Model hierarchy
│   ├── mobile/             # Mobile page objects
│   ├── types/              # TypeScript interfaces
│   └── utils/              # Logger (Winston), file handlers, etc.
├── tests/                  # Comprehensive test suites
│   ├── ui/                 # UI tests (smoke, regression, visual, e2e)
│   ├── api/                # API tests with schema validation
│   ├── ai/                 # AI/LLM tests
│   ├── security/           # Security/OWASP testing
│   ├── mobile/             # Mobile device emulation
│   ├── performance/        # Performance & Core Web Vitals
│   ├── a11y/               # Accessibility (WCAG 2.2 AA)
│   ├── data/               # Data management tests
│   └── flakiness/          # Flaky test detection & quarantine
├── Dockerfile              # Multi-stage build with Playwright base
├── docker-compose.yml      # Full stack: app, testsuite, allure, otel, jaeger
├── otel-collector-config.yml # OpenTelemetry collector configuration
├── package.json            # 50+ npm scripts for all testing scenarios
└── README.md               # This file
```

---

## Running Tests

```bash
# All tests
npm test

# By type
npm run test:ui              # UI tests
npm run test:api             # API tests
npm run test:ai              # AI/LLM tests
npm run test:mobile          # Mobile device emulation tests
npm run test:file-ops        # File operation tests
npm run test:security        # Security tests
npm run test:i18n            # Internationalization tests
npm run test:auth-matrix     # Multi-role auth tests
npm run test:contracts       # Consumer-driven contract tests (Pact)
npm run test:a11y            # Accessibility tests (WCAG 2.2 AA)
npm run test:performance     # Performance & Core Web Vitals tests

# By tag
npm run test:smoke           # @smoke
npm run test:regression      # @regression
npm run test:e2e             # @e2e
npm run test:visual          # @visual
npm run test:a11y            # @a11y
npm run test:performance     # @performance

# Debug modes
npm run test:headed          # Headed browser
npm run test:debug           # Playwright inspector

# Mutation testing
npm run test:mutation        # Stryker mutation analysis
```

### Per Environment

```bash
TEST_ENV=dev npm test
TEST_ENV=staging npm test
TEST_ENV=prod npm test
```

### Cross-Browser

```bash
ALL_BROWSERS=true npm test   # Chromium + Firefox + WebKit + mobile Chrome + mobile Safari
```

### Mobile Testing

```bash
npm run test:mobile                              # Pixel 5 + iPhone 13
npm run test:mobile -- --project mobile-chrome   # Android only
npm run test:mobile -- --project mobile-safari   # iOS only
MOBILE_TABLET=true npm run test:mobile           # Include iPad
npm run test:mobile -- --headed                  # Watch execution
```

---

## AI-Powered Testing

### LLM Client (AWS Bedrock)
Built on AWS Bedrock Converse API with Amazon Nova Pro as the default model. Supports chat, summarization, and embeddings (Titan Embed v2).

### AI Assertions
- **Semantic similarity** — cosine similarity on Titan embeddings with configurable threshold
- **Hallucination detection** — keyword-based grounding checks
- **AI locator fallback** — sends DOM snapshot to LLM to resolve broken selectors

### LLM-as-Judge
Multi-dimensional evaluation system with 6 built-in rubrics:

| Rubric | Use Case | Pass Threshold |
|---|---|---|
| Standard | General response quality | 3.5 |
| RAG Faithfulness | Grounding in source documents | 4.0 |
| Customer Support | Empathy, accuracy, actionability | 3.5 |
| Code Generation | Correctness, security, structure | 4.0 |
| Summarization | Coverage, conciseness, faithfulness | 3.5 |
| Toxicity & Safety | Hate speech, PII, prompt injection | 4.5 |

Scores across 6 dimensions: relevance, accuracy, coherence, safety, faithfulness, instruction adherence.

**Anti-narcissistic bias**: Primary model (Nova Pro) and judge model (Claude Haiku) use different architectures to prevent self-evaluation bias. Borderline scores (2–3) trigger automatic strict re-evaluation.

### Semantic Assertions API

```typescript
import { semanticExpect } from '@ai/semantic-assert';

await semanticExpect(response).toBeRelevantTo(prompt);
await semanticExpect(response).toBeGroundedIn(sourceContext);
await semanticExpect(response).toBeSafe();
await semanticExpect(response).toNotHallucinate(knownFacts);
await semanticExpect(response).toBeBetterThan(alternativeResponse);
await semanticExpect(response).toHaveScore({ min: 4.0, rubric: RUBRICS.CODE_GENERATION });
```

---

## MCP Server

Model Context Protocol server for AI-driven test orchestration. Supports both `stdio` and `SSE` transports.

```bash
npm run mcp:server           # stdio mode
npm run mcp:server:sse       # SSE mode on port 3001
npm run mcp:generate         # AI test generator client
```

**Available MCP tools:**
| Tool | Description | Example |
|---|---|---|
| `navigate` | Open URL, return title + screenshot | `navigate({url: "https://example.com"})` |
| `screenshot` | Full-page screenshot of any URL | `screenshot({url: "https://example.com"})` |
| `get_page_text` | Extract visible text from a page | `get_page_text({url: "https://example.com"})` |
| `run_test` | Execute a specific test file | `run_test({testFile: "tests/ui/login.spec.ts"})` |
| `list_tests` | List all available test specs | `list_tests({})` |
| `generate_test_from_description` | Generate complete test file from natural language feature description | `generate_test_from_description({description: "Test user login flow"})` |
| `generate_page_object` | Analyze a URL and generate a full Page Object Model class | `generate_page_object({url: "https://example.com/login"})` |
| `generate_api_test` | Generate API test suite from OpenAPI 3.0 specification | `generate_api_test({openapi_spec_url: "https://api.example.com/openapi.json"})` |
| `analyze_test_coverage` | Analyze Page Object Model and identify coverage gaps | `analyze_test_coverage({page_object_content: "export class LoginPage { ... }"})` |

**Test Generator Examples:**

Generate a complete login test:
```javascript
const result = await client.callTool('generate_test_from_description', {
  description: 'User logs in with email and password, verifies dashboard appears'
});
// Returns TypeScript test file ready to run with npm run test:smoke
```

Generate a Page Object Model by analyzing a live page:
```javascript
const pom = await client.callTool('generate_page_object', {
  url: 'https://orangehrm-demo.com/web/index.php/auth/login'
});
// Returns class LoginPage with all selectors and helper methods
```

Generate API test suite from spec:
```javascript
const apiTest = await client.callTool('generate_api_test', {
  openapi_spec_url: 'https://api.example.com/v1/openapi.json'
});
// Returns complete test suite validating all endpoints and schemas
```

Identify test coverage gaps:
```javascript
const gaps = await client.callTool('analyze_test_coverage', {
  page_object_content: pomClassContent
});
// Returns array of missing test scenarios and accessibility checks
```

---

## AI Agents

### Code Review Agent
```bash
npm run agent:review -- --file tests/ui/login.spec.ts
npm run agent:review -- --file tests/ui/login.spec.ts --slack   # post to Slack
```

### Test Healer Agent
Auto-detects broken locators in failed tests and suggests AI-powered fixes on a new git branch.
```bash
npm run agent:heal -- --test-result reports/playwright-report/results.json
```

### Suggestion Agent
Analyzes page objects + existing test coverage and generates 5–10 new test case suggestions.
```bash
npm run agent:suggest -- --page src/pages/employee.page.ts
```

### Jenkins Trigger Agent
```bash
npm run agent:jenkins -- --job playwright-tests --params ENV=staging
npm run agent:jenkins -- --job playwright-tests --build 42
npm run agent:jenkins -- --job playwright-tests --build 42 --logs
```

### Jira Sync Agent
Auto-creates/updates/closes Jira bugs from test results. Attaches JUnit XML and links test executions.
```bash
npm run agent:jira -- --build latest
npm run agent:jira -- --junit reports/playwright-report/junit.xml --build 42
npm run agent:jira -- --junit reports/playwright-report/junit.xml --dry-run
```

### Run-and-Report Agent
Full workflow: trigger Jenkins → stream console → download artifacts → sync to Jira → notify Slack.
```bash
npm run agent:run-and-report
npm run agent:run-and-report -- --suite smoke --jira-key SCRUM-5
npm run agent:run-and-report -- --suite regression --jira-key SCRUM-5 --dry-run
```

### Slack Bot Agent
```bash
npx ts-node agents/slack-bot-agent.ts --report reports/playwright-report/results.json
```

---

## Contract Testing

Consumer-driven contract testing using

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [nshportun](https://github.com/nshportun)
- **Source:** [nshportun/BestTester](https://github.com/nshportun/BestTester)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-nshportun-besttester
- Seller: https://agentstack.voostack.com/s/nshportun
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.