Skip to content

TESTING.md

Latest commit

 

History

History
108 lines (71 loc) · 4.58 KB

TESTING.md

File metadata and controls

108 lines (71 loc) · 4.58 KB
Raw

Testing

This project keeps default automated checks local and deterministic. Anything that needs external credentials, real downstream agents, production storage, deployment infrastructure, or conformance/certification claims is outside the current pre-M4 test surface.

Quick Checks

Run these before changing runtime behavior:

go test ./...
ubs --only=golang --ci .

Inside the pinned development shell, the equivalent commands are:

nix develop -c just test
nix develop -c just ubs-ci

ubs-ci is a static warning scan. It is useful review input, but it is not currently a hard correctness gate because the current Go scan includes conservative warnings and informational heuristics. Critical UBS findings should be treated as blockers unless they are proven false positives and documented here.

Unit Tests

go test ./... covers the fast, local-only test suite:

  • config loading and contract validation;
  • A2A command metadata mapping;
  • SQLite Bridge Store reservation, duplicate handling, restart persistence, and snapshot writes;
  • JSON-RPC API projection and public-field scrubbing;
  • mock reviewer idempotency and task-polling behavior;
  • Temporal workflow branch scheduling, partial-failure policy, bundle construction, and activities through SDK test environments.

These tests do not require Temporal server, network credentials, external A2A agents, Postgres, artifact proxy storage, or production security configuration.

M1 Smoke

Run:

nix develop -c just smoke-m1-temporal

This starts a local Temporal dev server, Bridge API, worker, and one mock reviewer agent, then runs cmd/a2a-temporal-smoke against the JSON-RPC endpoint. It verifies the one-branch demo path:

  • submitted -> working -> completed is observable;
  • terminal review_bundle_v1 is present;
  • duplicate same-key/same-body returns the same task;
  • same-key/different-body is rejected;
  • cleanup stops local child processes.

The recipe uses fixed local demo ports, so do not run it in parallel with other smoke recipes.

M2 Smoke

Run:

nix develop -c just smoke-m2-parallel

This starts local Temporal, Bridge API, worker, one success mock reviewer, and one permanent-failure mock reviewer. It validates both configured M2 smoke profiles:

  • m2-success: three branches complete in deterministic contract order with degraded=false;
  • m2-optional-failure: required branches succeed, optional ops fails safely, and the parent completes with degraded=true.

The smoke is mock-only and makes no production readiness, certification, conformance, or arbitrary downstream exactly-once claim.

Recording Demo

For an interactive portfolio recording:

nix develop -c just demo-m2-recording
nix develop -c just demo-m2-client

demo-m2-recording keeps a local mock-only stack running until interrupted and prints the Agent Card, JSON-RPC endpoint, Temporal UI URL, and client command. demo-m2-client submits one recording-friendly task to the running stack. This flow is for demonstration ergonomics, not default CI.

UBS Status

The current UBS scan has zero critical findings. The remaining warning after the low-risk mutex cleanup is a conservative time.NewTicker without Stop finding in cmd/a2a-temporal-smoke; the code already defers ticker.Stop() immediately after construction in pollTerminalTask.

Other UBS informational items are not currently treated as defects without code review confirmation. Examples include JSON decoder hardening, context parameter position, deferred close error checks, and t.Parallel() suggestions.

CI

GitHub Actions runs:

  • go test ./...;
  • command binary builds for cmd/a2a-temporal and cmd/a2a-temporal-smoke;
  • non-blocking ubs --only=golang --ci ..

The M1 and M2 smoke recipes are available in the same workflow through manual workflow_dispatch. They are kept manual because they start local Temporal and fixed-port demo services.

Future M4 Scenarios

M4 approval, continuation, and cancellation are intentionally not implemented in this hardening chunk. When M4 begins, add tests for:

  • entering input-required only when approval is enabled by contract/input;
  • valid approval signal completing exactly once;
  • invalid approval schema rejection;
  • duplicate continuation same-key/same-body replay;
  • conflicting continuation same-key/different-body rejection;
  • continuation against terminal tasks not restarting workflows;
  • parent CancelTask mapping to Temporal cancellation;
  • cancellation races with branch completion;
  • truthful downstream cancellation metadata for unsupported, not-attempted, failed, and succeeded outcomes.