Skip to content

Add Playwright for E2E and Accessibility Testing #152

Description

@jbouder

Summary

Add Playwright to frontend/ for browser-level end-to-end coverage of the API key management flows, plus automated accessibility checks via @axe-core/playwright. Adds a new e2e-frontend job to .github/workflows/test.yaml so both suites gate every PR alongside the existing Vitest, Go, and Helm jobs.

Motivation

  • Current frontend coverage is Vitest + Testing Library only (src/**/*.test.tsx) — jsdom, no real browser, no real navigation, no real Keycloak bootstrap path.
  • The multi-step key lifecycle (create → one-time secret reveal in KeyCreatedDialog → revoke) spans several components and is only verified in isolation today.
  • src/auth/keycloak.ts already exposes a window.__PW_E2E_AUTH__ injection hook explicitly reserved for Playwright — the auth seam exists but nothing uses it.
  • No accessibility testing exists at all; the app ships custom dialogs, a theme toggle, and Base UI/Radix primitives whose focus trapping, labeling, and contrast are currently unverified.
  • Both light and dark themes (ThemeProvider) plus the branding/theming work merged in Branding Support #145 need contrast regression protection.
  • Real-browser tests catch class of bugs jsdom cannot: focus management, scroll locking, portal rendering, CSS-dependent visibility.

Acceptance Criteria

  • @playwright/test and @axe-core/playwright added to frontend/devDependencies
  • frontend/playwright.config.ts created with a webServer block that builds/serves the app (npm run build && npm run preview) and a baseURL matching the preview port
  • Chromium project configured at minimum; trace: "on-first-retry" and retries set for CI
  • E2E specs live under frontend/e2e/ and are excluded from the Vitest include glob in vite.config.ts so npm test does not pick them up
  • Auth is stubbed via page.addInitScript setting window.__PW_E2E_AUTH__ (no live Keycloak in CI)
  • Backend /api calls stubbed with Playwright page.route fixtures — no live key-manager dependency in CI
  • E2E spec: keys list renders, including the empty state
  • E2E spec: create a key via CreateKeyDialog, assert the secret is shown once in KeyCreatedDialog and is copyable
  • E2E spec: revoke a key via KeyRowActionsRevokeKeyDialog, assert the row updates
  • E2E spec: API failure path renders ErrorBanner
  • E2E spec: theme toggle switches light/dark and persists across reload
  • A11y spec runs AxeBuilder against the main view and each open dialog, asserting zero violations for wcag2a, wcag2aa, and wcag21aa tags
  • A11y assertions run in both light and dark themes (color-contrast included, not disabled)
  • Any pre-existing violations are either fixed or recorded in a documented, time-boxed exclusion list rather than silently suppressed
  • npm run e2e and npm run e2e:ui scripts added to frontend/package.json
  • e2e-frontend job added to .github/workflows/test.yaml: checkout → actions/setup-node (using frontend/.node-version, npm cache) → npm cinpx playwright install --with-deps chromiumnpm run e2e
  • Playwright browser binaries cached in CI (keyed on the resolved Playwright version) to keep job time reasonable
  • playwright-report/, test-results/, and blob-report/ added to frontend/.gitignore
  • HTML report uploaded via actions/upload-artifact on failure, with the same pinned-SHA convention used by the other workflow steps
  • Biome config updated so frontend/e2e/ lints clean under npm run check
  • README.md (or docs/) documents how to run E2E locally, including running against a real key-manager with VITE_DEV_NO_AUTH=true

Out of Scope

  • Component-level and hook-level unit tests — already covered by Vitest + Testing Library in src/
  • Go test coverage for operator/ and key-manager/ — handled by the existing test-operator and test-key-manager jobs
  • Live Keycloak login flow testing (real IdP redirect + PKCE); this issue stubs auth at the __PW_E2E_AUTH__ seam
  • Cross-browser matrix (Firefox/WebKit) and mobile viewports — Chromium only for now, expandable later
  • Visual regression / screenshot snapshot testing
  • Manual accessibility audit (screen reader walkthroughs, keyboard-only review) — axe catches automated violations only, roughly 30–40% of WCAG issues
  • Load or performance testing of the vLLM serving path

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Fields

    Priority

    Medium

    Start date

    None yet

    Target date

    None yet

    Size

    M

    Projects

    Status
    Ready

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions