E2E Testing Harness with Playwright + code-server

[IanMayo]

Summary

We need full end-to-end (E2E) test coverage for our VS Code extension as it will be deployed inside a code-server container (including Fly.io). This issue proposes a Playwright-based harness that drives the code-server web UI in a container, simulating analyst workflows.

Why

• Validate the extension in the same environment analysts will use (browser + code-server).
• Catch UI-level regressions not visible in extension-host tests.
• Allow reproducible testing in CI/CD (e.g., GitHub Actions).

Proposed Approach

1. Test Docker Image

Create a test-specific Dockerfile extending code-server:

FROM codercom/code-server:latest

# Install our extension (assumes .vsix built in CI step)
COPY ./dist/our-extension.vsix /tmp/
RUN code-server --install-extension /tmp/our-extension.vsix

# Copy in test workspace data (e.g. GeoJSON files)
COPY ./test-data /home/coder/workspace

# Disable auth for automated testing
ENV PASSWORD=""
ENV AUTH=none

EXPOSE 8080
CMD ["code-server", "--bind-addr", "0.0.0.0:8080", "--disable-telemetry", "--accept-server-license-terms", "/home/coder/workspace"]

2. Local Startup

For dev runs:

docker build -t vscode-test .
docker run -d -p 8080:8080 --name vscode-test vscode-test

3. Playwright Config

playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  use: {
    baseURL: 'http://localhost:8080',
    browserName: 'chromium',
  },
});

4. Example Test

tests/e2e/basic.spec.ts:

import { test, expect } from '@playwright/test';

test('Extension sidebar shows up', async ({ page }) => {
  await page.goto('/?folder=/home/coder/workspace');

  await page.waitForSelector('.monaco-workbench');

  // Open extension view (adjust selector to match actual sidebar icon/title)
  await page.click('[title="Our Extension"]');

  // Verify custom panel renders correctly
  await expect(page.locator('text=My Custom Panel')).toBeVisible();
});

5. CI Integration

Use GitHub Actions with a service container:

jobs:
  e2e:
    runs-on: ubuntu-latest
    services:
      vscode:
        image: vscode-test
        ports:
          - 8080:8080
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test

6. Test Data

• Store small GeoJSON/REP/DPF files in test-data/ folder.
• Preload them into container at /home/coder/workspace.
• Tests can open them via the VS Code file explorer or direct URLs.

7. Coverage Strategy

• Playwright + code-server: simulate analyst workflows (UI clicks, file opens, sidebars, maps).
• @vscode/test-electron: cover API-level correctness in headless runs.
• Unit tests: cover algorithmic logic.

---

Tasks

• Create test-data/ folder with representative files.
• Add Dockerfile for test image.
• Add Playwright config + first test.
• Wire up CI job for automated runs.
• Extend tests to cover common analyst workflows.

Acceptance Criteria

• Running npx playwright test against a local container validates extension sidebar rendering.
• CI runs the same tests automatically on PRs.
• At least one test opens a GeoJSON file and verifies that the extension renders expected output.