Skip to content

Latest commit

 

History

History
428 lines (332 loc) · 16.6 KB

File metadata and controls

428 lines (332 loc) · 16.6 KB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.


Project Overview

Enterprise-grade API test automation framework for the ENTERPRISE/API contact center platform. Tests are written in Node.js using Mocha + Supertest + Chai. The framework includes AI-powered test generation from OpenAPI specs, Allure reporting, and TestRail integration.

  • Test target: https://flake-api.API.co (staging environment)
  • Account: flake (ACCOUNT_NAME)
  • Branch conventions: master is main; feature branches go to PRs

Repository Structure

enterprise-api-template/
├── api/                         # OpenAPI specs & flows YAML files
│   ├── openapi.yaml             # API spec
│   ├── dummyjson-flows.yaml     # DummyJSON-specific E2E flows
│   └── flows.example.yaml       # Example flows template
├── .claude/                     # Claude Code skills (slash commands)
│   └── skills/                  # SKILL.md files — generate-tests, enhance-tests, push-to-testrail, generate-flows
├── config/                      # Config (index.js)
├── helpers/                     # Shared test utilities
│   ├── auth.js                  # Auth token helpers (DummyJSON JWT)
│   ├── request.js               # CRUD HTTP wrappers (get, getAll, create, update, deleteOne…)
│   ├── assertions.js            # HTTP status assertion helpers (toCreate, to412, to404, etc.)
│   ├── data-factory.js          # Faker.js data generation
│   └── cleanup.js               # Test data cleanup
├── scripts/
│   └── push-to-testrail.js      # TestRail sync script
├── tests/                       # Main test suite
├── tests/smoke/                 # Smoke tests
├── tests/regression/            # Regression tests
├── tests/integration/           # Integration flows
├── tests/data-driven/           # DDT tests
├── tests/generated/             # AI-generated test files
├── tests/performance/           # k6 performance suite (smoke / load / stress profiles)
├── .github/workflows/           # GitHub Actions CI/CD
├── .env                         # Credentials & service URLs (never commit)
├── package.json
└── CLAUDE.md                    # This file

NPM Scripts

Script Purpose
npm test Run all tests (mocha-multi + Allure reporter)
npm run test:cdr Run CDR tests only
npm run test:generated Run tests in tests/generated/
npm run test:regression Run tests tagged [regression]
npm run allure:run Run tests → generate → open Allure report
npm run allure:generate Generate Allure HTML from results
npm run allure:open Open generated Allure report
npm run generate:tests Generate test file from OpenAPI spec (requires skill script)
npm run generate:flows Generate flow tests from flows.yaml (requires skill script)
npm run enhance:tests Enhance generated test with business-rule tests (requires skill script)
npm run push:testrail Push test cases to TestRail
npm run push:testrail:dry Dry-run TestRail push (safe preview)
npm run perf:smoke k6 smoke — 1 VU, 30s sanity check
npm run perf:load k6 load — 10 VUs, 60s baseline
npm run perf:stress k6 stress — ramp 5 → 50 VUs
npm run perf:report Generate HTML reports from perf results
npm run perf:all Run all perf profiles then generate report

Run a single test file:

npx mocha tests/users.js --timeout 90000 --exit

Run tests matching a name pattern:

npx mocha tests/users.js --timeout 90000 --exit --grep '\[smoke\]'

Note: scripts/generate-tests.js, scripts/generate-flows.js, and scripts/enhance-tests.js are created by the Claude Code skills (/generate-tests, /generate-flows, /enhance-tests). Use the slash commands rather than the npm scripts unless those files already exist.


Key Dependencies

  • mocha — Test framework
  • chai — Assertions (expect style)
  • supertest — HTTP request/assertion library
  • @faker-js/faker — Test data generation
  • chai-openapi-response-validator — OpenAPI contract validation (expect(res).to.satisfyApiSpec)
  • allure-mocha + mocha-multi — Dual-reporter (spec + Allure)
  • dotenv — Loads .env
  • @anthropic-ai/sdk — Claude API used internally by generation scripts
  • js-yaml — YAML parsing (for flows.yaml)
  • moment / moment-random — Date generation
  • crypto-js — Encryption utilities
  • lodash — Utility functions

Test Writing Conventions

File anatomy

require('dotenv').config()
const { expect } = require('chai')
const admins   = require('../../helpers/auth')
const go       = require('../../helpers/request')
const attempt  = require('../../helpers/assertions')

// context() is a Mocha alias for describe() — both are used in this codebase
context('<Resource>', () => {
    describe('<Resource> Tests', () => {
        let accessToken

        before('Prepare for Tests', async () => {
            accessToken = await admins.getAdminToken()
        })

        it('[smoke] [regression] Create a <resource>...', async () => { ... })
        it('[regression] Update a <resource>...', async () => { ... })
        it('[regression] Validate missing required field returns 412...', async () => { ... })
    })
})

Tags

  • [smoke] — Basic happy-path, run in smoke pass
  • [regression] — Full regression suite; also used by TestRail push to mark High priority

Lifecycle pattern (CRUD test)

  1. Get admin token in before() using admins.getAdminToken()
  2. Create resource with go.create()
  3. Assert 201 + expected fields
  4. Update resource with go.update()
  5. Assert 200 + update succeeded
  6. Always delete the resource at end using go.deleteOne() — keep environment clean

Error status assertions

Status Helper Meaning
412 attempt.toCreate412() etc. Validation error (missing field, bad enum, wrong type)
404 attempt.toGet404() etc. Resource not found
401 attempt.toGet401() etc. Unauthorized / no token
409 attempt.toCreate409() Duplicate / conflict

Data generation

const { faker } = require('@faker-js/faker')
faker.person.firstName()
faker.person.lastName()
faker.internet.email(firstName, lastName, 'ENTERPRISE.io')
faker.string.uuid()
faker.string.alpha({ length: 10 })
faker.datatype.boolean()
process.env.USER_PASSWORD   // shared password for test users

No try/catch

Do not wrap test bodies in try/catch. Let Mocha capture and report errors natively.


Helpers Reference

helpers/auth.js

Function Description
getAdminToken() Returns admin access token (JWT)
loginAsUser(user, pass) Login as specific user

helpers/request.js (aliased as go)

Function Description
get(path, token) GET — returns raw response
getAll(path, token) GET list — returns raw response
create(path, data, token) POST — returns raw response
update(path, data, token) PUT — returns raw response
deleteOne(path, token) DELETE — returns raw response

helpers/assertions.js

Wraps raw supertest assertions into semantic helpers. Pattern: toVerb[Status].

Important: These helpers make the HTTP call internally and return res.body.error (not the response object). They are not wrappers around an existing response — pass module, id/data, and token directly.

Examples: toGet(module, id, token), toCreate(module, data, token), toUpdate(module, data, token), toDelete(module, id, token), toGet404(module, id, token), toCreate412(module, data, token), toUpdate412(module, data, token), toCreate409(module, data, token), toGet401(module, id, token), toBlock(module, id, token), toUnblock(module, id, token).

helpers/checks.js

Query-param and field validation utilities used for systematic parameter testing. limitParameter(endpoint), skipParameter(endpoint), searchForXXXString(endpoint), searchByParam(endpoint, param, value), orderByParamAsc/Desc(endpoint, param).


AI-Powered Test Generation Skills

Invoke via slash commands in Claude Code:

Skill What it does
/generate-tests Generate tests/generated/<tag>.js from an OpenAPI spec (spec mode) or a flows.yaml (flows mode)
/enhance-tests Add missing business-rule / state-machine / cross-resource tests to a generated file
/generate-flows Read a Linear project/milestone/tickets and write api/<tag>-flows.yaml for E2E test generation
/push-to-testrail Sync test cases from a .js file or directory to TestRail

SKILL.md files live in .claude/skills/<skill-name>/SKILL.md — read them before running a skill for full argument/options details.

Typical workflow

/generate-flows --linear <Linear-URL-or-issue-keys> --tag <name>
        ↓  reads Linear project deeply → writes api/<tag>-flows.yaml
/generate-tests --flows api/<tag>-flows.yaml
        ↓  writes tests/generated/<tag>.js
/enhance-tests tests/generated/<tag>.js
        ↓  adds business-rule & cross-resource tests in-place
/push-to-testrail --file tests/generated/<tag>.js
        ↓  syncs to TestRail

You can also run spec mode directly without flows:

/generate-tests api/openapi.yaml --tag <name>

api/flows.yaml format

flows:
  - name: "Agent full lifecycle"
    tag: agents
    description: "Create, read, update, block, unblock, delete an agent"
    linear_refs:         # traceability — Linear issue keys this flow covers
      - ENTERPRISE-123
    steps:
      - id: create_agent
        method: POST
        path: /admin/agents/
        contentType: application/x-www-form-urlencoded   # only when not JSON
        body:
          name: "{{faker.firstName}}"
          username: "{{faker.email}}"
          password: "{{env.USER_PASSWORD}}"
        capture:
          agentId: content.id          # save for subsequent steps
        expect:
          status: 200
          body.error: null

      - id: read_agent
        method: GET
        path: /admin/agents/{{capture.agentId}}
        expect:
          status: 200
          body.content.id: "{{capture.agentId}}"

      - id: cleanup
        method: DELETE
        path: /admin/agents/{{capture.agentId}}
        expect:
          status: 200

Interpolation tokens: {{faker.firstName}}, {{faker.lastName}}, {{faker.email}}, {{faker.uuid}}, {{faker.alphaNum(N)}}, {{faker.number(N)}}, {{env.VAR_NAME}}, {{capture.varName}}

Named flows files follow the pattern api/<tag>-flows.yaml (e.g. api/broadcast-flows.yaml). The generic api/flows.yaml is the shared fallback.


Environment Variables (.env)

.env is git-ignored. Never commit it.

Variable Description
BASEURL Main API base URL (https://flake-api.API.co)
API_KEY API key for monitoring/key-auth endpoints
SUPER_ADMIN_USERNAME Super admin login (ENTERPRISE@API-telecom.com)
SUPER_ADMIN_PASSWORD Super admin password (encrypted)
USER_PASSWORD Shared password for created test users (Testtest123!@#)
TEST_AGENT_USERNAME Pre-existing test agent login
TEST_AGENT_PASSWORD Test agent password
TEST_ADMIN_USERNAME Pre-existing test admin login
BLOCK_AGENT_USERNAME Pre-existing blocked agent login
ACCOUNT_NAME Account name (flake)
ACCOUNT_NAME_MFA_EMAIL Account with email MFA (rainbow)
ACCOUNT_NAME_MFA_SMS Account with SMS MFA (ENTERPRISE-load-test)
STRAPI_URL Strapi CMS URL for billing management
STRAPI_USERNAME/PASSWORD/ID Strapi credentials
DIRECTORY_URL Directory service URL
AUTH_URL / ZAUTH_URL Auth service URLs
TESTRAIL_URL TestRail instance URL
TESTRAIL_USERNAME TestRail username
TESTRAIL_API_KEY TestRail API key
TESTRAIL_PROJECT_ID TestRail project ID (1)
TESTRAIL_SUITE_ID TestRail suite ID (1)

Flows Files

File Resource
api/flows.yaml Generic shared E2E flows
api/flows.example.yaml Template/reference for writing new flows files

TestRail Integration

Script: scripts/push-to-testrail.js

# Always dry-run first
node scripts/push-to-testrail.js --file tests/generated/users.js --dry-run

# Push a single file
node scripts/push-to-testrail.js --file tests/generated/users.js

# Push entire directory
node scripts/push-to-testrail.js --dir tests/generated/
  • Creates a section per resource tag; creates or updates (never duplicates) test cases
  • Tests tagged [regression]High priority in TestRail
  • Requires: TESTRAIL_URL, TESTRAIL_USERNAME, TESTRAIL_API_KEY, TESTRAIL_PROJECT_ID

Performance Testing

The framework includes a k6-based performance suite in tests/performance/. It runs independently of Mocha and is excluded from npm test.

Tool: k6 (must be installed separately — brew install k6)

Structure

tests/performance/
├── smoke.js              # 1 VU, 30s — sanity check
├── load.js               # 10 VUs, 60s — baseline throughput & latency
├── stress.js             # ramp 5 → 50 VUs over 2min — concurrency limits
├── perf-run.js           # Node.js launcher — fetches auth token, then calls k6
├── perf-report.js        # Generates HTML reports from perf-results/*.json
├── lib/
│   ├── auth.js           # Reads PERF_TOKEN env var (set by perf-run.js)
│   ├── http.js           # k6 HTTP helpers (get, post, put, del) with auto checkResponse()
│   ├── checks.js         # checkResponse(), checkStatus(), checkField()
│   └── thresholds.js     # Shared p95/p99/error-rate threshold configs
├── scenarios/
│   ├── agents.js         # listAgents(), agentLifecycle() (create + delete)
│   ├── users.js          # listUsers(), userLifecycle()
│   └── queues.js         # listQueues()
└── perf-results/         # JSON + HTML reports (git-ignored)

NPM Scripts

Script What it runs
npm run perf:smoke 1 VU, 30s — sanity check before heavier runs
npm run perf:load 10 VUs, 60s — baseline load across agents/users/queues
npm run perf:stress Ramp 5 → 50 VUs — stress test
npm run perf:report Generate HTML reports from all saved JSON results
npm run perf:all Run smoke → load → stress → report in sequence

Auth flow: perf-run.js uses helpers/admins.superAdminToken() (same as Mocha tests) to fetch a token, then passes it to k6 via PERF_TOKEN env var — no credential duplication.

Thresholds (production standards)

Metric Target
p95 response time < 1000ms
p99 response time < 2000ms
Error rate < 0.1%
Check pass rate > 99%

Per-endpoint thresholds are tracked individually for /admin/agents, /admin/users, /admin/queues/.

Known baseline (staging): avg ~755ms, p95 ~1064ms (currently failing p95 threshold). /admin/agents and /admin/queues/ consistently > 500ms — flagged for investigation.

Adding a new scenario

  1. Create tests/performance/scenarios/<resource>.js — export list<Resource>() and optionally <resource>Lifecycle()
  2. Import and wire into smoke.js (read-only) and load.js (read + write scenarios)
  3. Add per-endpoint threshold entries in lib/thresholds.js

CI/CD

GitHub Actions workflow: workflows/run-api-tests.yml

  • Runs on push/PR
  • Executes npm test
  • Generates Allure report

Allure results: allure-results/ (git-ignored) Allure report: allure-report/ (git-ignored)


Common Pitfalls & Rules

  1. Never add try/catch in test bodies — Mocha handles errors natively.
  2. Always clean up — Delete every resource created in a test (in the same it() block or after()).
  3. Use [regression] tag on any test that should be in the full regression suite.
  4. Use [smoke] tag for basic happy-path create/read tests.
  5. Do not hardcode IDs or URLs — use .env values via process.env.*.
  6. Generated tests go in tests/generated/ — not in tests/ directly.
  7. Do not commit .env — it is git-ignored and contains real credentials.
  8. No extra comments or docstrings on unchanged code — only comment non-obvious logic.
  9. helpers/assertions.js wraps status checks — use it instead of raw expect(res.status).to.eql(200).
  10. helpers/commonRequests.js is the CRUD layer — don't call supertest directly for standard CRUD.