Skip to content

motus-v1.0.11

Choose a tag to compare

@github-actions github-actions released this 02 May 23:35
· 47 commits to main since this release

Code coverage feature: per-test JavaScript and CSS coverage collected directly through CDP, source-map remapping back to original sources, console/HTML/Cobertura reporters, and CI-gated thresholds. Test-level retry on transient CDP disconnects and a transport-level hang fix round out the release.

Improvements

  • Code coverage collection over CDP - JavaScript coverage is collected per-page through Profiler.startPreciseCoverage (call-count plus detailed function ranges) and CSS rule-usage coverage through CSS.startRuleUsageTracking, both gated behind a new MotusCapabilities.CodeCoverage flag so BiDi sessions short-circuit cleanly with a diagnostic message instead of failing. The built-in CoverageCollector plugin enables the domains on page creation, takes a snapshot on page close, fetches script source through Debugger.getScriptSource, and pushes a CoverageData record into the per-test CoverageSink. Disabled by default; enabled via LaunchOptions.Coverage, the MOTUS_COVERAGE_ENABLE env var, the coverage.enable config key, or the CLI --coverage flag.
  • Per-test and per-run coverage aggregation - CoverageAggregator.SummarizeScript derives line-level stats from raw V8 byte ranges by mapping offsets to lines through the script source. MergeScripts, MergeStylesheets, and MergeOriginalFiles combine snapshots from many tests into a single aggregated CoverageData for the run, taking the max-coverage range for any overlapping ranges so a file covered by one test plus the same file covered by another test reports the union, not double-counted hits.
  • Source-map remapping back to original sources - When a coverage snapshot points at a bundled or minified script with a reachable source map (inline data URL or //# sourceMappingURL= reference), SourceMapResolver fetches the map (data URL inlined or HTTP-fetched with a per-process cache), SourceMapParser decodes the mappings via the new Vlq decoder, and CoverageRemapper walks the script's hit ranges and rewrites them onto the original source files. Resulting OriginalFileCoverage entries surface in CoverageData.OriginalFiles and feed both the report renderers and threshold evaluation, so a coverage report against app.min.js shows hits against Login.tsx and Header.tsx instead.
  • Coverage reporters and the ICoverageReporter plugin interface - CoverageConsoleReporter prints a per-file summary table colour-coded by coverage percentage. CoverageHtmlReporter emits a static HTML site with an index page, file tree, and per-file source views with green/red line-by-line highlighting. CoberturaReporter writes an XML document with <coverage> line-rate / lines-covered / lines-valid attributes plus packaged <class> entries with line-level hit counts, suitable for ingestion by Codecov, Azure DevOps, GitLab, and Jenkins. All three flow through the new CoverageReporterFactory, which parses --coverage specs and supports repeating the flag for multiple formats in one run. Third-party reporters opt in by implementing ICoverageReporter alongside IReporter: OnCoverageCollectedAsync fires per test, OnCoverageRunEndAsync fires once with the aggregated data — the same opt-in pattern as IAccessibilityReporter and IPerformanceReporter.
  • Coverage thresholds with CI failure on miss - CoverageThresholds.Evaluate compares the run's aggregated stats against coverage.js.lines, coverage.js.functions, and coverage.css.rules from motus.config.json (or LaunchOptions.Coverage). When a threshold is set and the run's percentage falls below it, the CLI prints Coverage threshold failed: <metric> <actual>% < <threshold>% and returns a non-zero exit code via TestRunResult.CoverageThresholdsFailed, so CI fails the build.
  • Coverage panel in the visual runner - The Blazor visual runner gains a CoveragePanel alongside the existing timeline and console panels. The panel reads from the new ICoverageService/CoverageService, which TestExecutionService feeds with the same snapshots that flow into the per-test sink. Per-test coverage is browsable inline as tests complete; aggregated run-end coverage shows in a summary view.
  • --coverage CLI flag with multi-format output and friendly errors - motus run --coverage console --coverage html:./out --coverage cobertura:./coverage.xml enables coverage and emits all three formats from a single run. Bare --coverage defaults to console. The flag flips MOTUS_COVERAGE_ENABLE=true for the run so the CoverageCollector plugin engages without requiring config-file changes. Invalid specs now emit a clean one-line error to stderr instead of dumping a stack trace through the System.CommandLine pipeline: --coverage html (missing target) tells the user the format requires <dir> and gives an example; --coverage cobertura does the same for <path>; --coverage console:foo rejects the spurious target; --coverage xml lists the supported formats; --coverage html: (empty target after the colon) is rejected with the same guidance. CLI returns exit code 1 for any of these so CI fails fast.
  • --retries N for transient CDP disconnects - Re-runs a failing test up to N additional times when (and only when) the failure chain contains CdpDisconnectedException or MotusTargetClosedException. The check walks InnerException, so wrapped failures still qualify. Non-transient failures (assertion errors, timeouts, real test bugs) are never retried, so flake recovery doesn't mask regressions. Each retry runs the entire test fresh: fresh test instance, fresh TestInitialize, fresh browser context, fresh WebSocket. Per-test sink data from a discarded attempt is dropped before the next attempt begins, so coverage/a11y/perf totals don't double-count. A [RETRY] <test> (CDP disconnect, attempt N/total) line is written to stderr for every retry, formatted, indented, and coloured to match the [PASS]/[FAIL] lines so it reads as part of the test stream.
  • Tracing serialization across browser sessions - CDP Tracing.start is browser-wide: only one trace can be active per browser process at a time. When multiple BrowserContexts share a browser (e.g. parallel workers under a shared-browser fixture), simultaneous StartAsync calls used to collide and one would fail with a CDP error. Tracing now coordinates through a ConditionalWeakTable<IMotusSession, SemaphoreSlim> keyed on the browser session, so concurrent starts queue rather than collide. The gate is held until StopAsync (or until StartAsync itself errors out, at which point it rolls back state and releases). Stale tracingComplete continuations from a previous run are also drained and their pumps cancelled at the top of StartAsync, preventing one run's dataCollected events from poisoning the next.

Bug Fixes

  • Profiler.getScriptSource doesn't exist; coverage was always 0/0 lines - The original coverage teardown called Profiler.getScriptSource, which is not a CDP method — script-source retrieval lives on the Debugger domain. Every call threw, the catch block logged the error, source stayed empty, and CoverageAggregator.SummarizeScript reported TotalLines = 0 for every script, collapsing every percentage to 0.0% regardless of how much code actually ran. Fixed by enabling the Debugger domain alongside Profiler in OnPageCreatedAsync and switching the source fetch to Debugger.getScriptSource. A best-effort Debugger.disable runs after Profiler.stopPreciseCoverage for symmetry.
  • Post-disconnect 60-second-per-send hang in CdpTransport - SendRawAsync only checked _disposed, not whether the receive loop had already observed a disconnect. After the receive loop exited and faulted in-flight requests, new sends were still accepted, the bytes went out, and each one waited the full 60-second CommandTimeout for a response that could never arrive. With coverage enabled this turned post-disconnect cleanup paths (one Profiler.takePreciseCoverage plus one Debugger.getScriptSource per script plus stopPreciseCoverage plus Debugger.disable) into multi-minute hangs that required Ctrl-C to recover from, masquerading as the test runner having frozen. Fixed by adding a _disconnected volatile flag set inside the receive loop's disconnect paths (clean close, exception, and dispose) through a single MarkDisconnectedAndFault helper. SendRawAsync now checks _disposed || _disconnected at the top and throws CdpDisconnectedException immediately. A regression test asserts that a send issued after an observed disconnect throws within two seconds rather than the prior sixty.