motus-v1.0.11
·
47 commits
to main
since this release
Code coverage feature: per-test JavaScript and CSS coverage collected directly through CDP, source-map remapping back to original sources, console/HTML/Cobertura reporters, and CI-gated thresholds. Test-level retry on transient CDP disconnects and a transport-level hang fix round out the release.
Improvements
- Code coverage collection over CDP - JavaScript coverage is collected per-page through
Profiler.startPreciseCoverage(call-count plus detailed function ranges) and CSS rule-usage coverage throughCSS.startRuleUsageTracking, both gated behind a newMotusCapabilities.CodeCoverageflag so BiDi sessions short-circuit cleanly with a diagnostic message instead of failing. The built-inCoverageCollectorplugin enables the domains on page creation, takes a snapshot on page close, fetches script source throughDebugger.getScriptSource, and pushes aCoverageDatarecord into the per-testCoverageSink. Disabled by default; enabled viaLaunchOptions.Coverage, theMOTUS_COVERAGE_ENABLEenv var, thecoverage.enableconfig key, or the CLI--coverageflag. - Per-test and per-run coverage aggregation -
CoverageAggregator.SummarizeScriptderives line-level stats from raw V8 byte ranges by mapping offsets to lines through the script source.MergeScripts,MergeStylesheets, andMergeOriginalFilescombine snapshots from many tests into a single aggregatedCoverageDatafor the run, taking the max-coverage range for any overlapping ranges so a file covered by one test plus the same file covered by another test reports the union, not double-counted hits. - Source-map remapping back to original sources - When a coverage snapshot points at a bundled or minified script with a reachable source map (inline data URL or
//# sourceMappingURL=reference),SourceMapResolverfetches the map (data URL inlined or HTTP-fetched with a per-process cache),SourceMapParserdecodes the mappings via the newVlqdecoder, andCoverageRemapperwalks the script's hit ranges and rewrites them onto the original source files. ResultingOriginalFileCoverageentries surface inCoverageData.OriginalFilesand feed both the report renderers and threshold evaluation, so a coverage report againstapp.min.jsshows hits againstLogin.tsxandHeader.tsxinstead. - Coverage reporters and the
ICoverageReporterplugin interface -CoverageConsoleReporterprints a per-file summary table colour-coded by coverage percentage.CoverageHtmlReporteremits a static HTML site with an index page, file tree, and per-file source views with green/red line-by-line highlighting.CoberturaReporterwrites an XML document with<coverage>line-rate / lines-covered / lines-valid attributes plus packaged<class>entries with line-level hit counts, suitable for ingestion by Codecov, Azure DevOps, GitLab, and Jenkins. All three flow through the newCoverageReporterFactory, which parses--coveragespecs and supports repeating the flag for multiple formats in one run. Third-party reporters opt in by implementingICoverageReporteralongsideIReporter:OnCoverageCollectedAsyncfires per test,OnCoverageRunEndAsyncfires once with the aggregated data — the same opt-in pattern asIAccessibilityReporterandIPerformanceReporter. - Coverage thresholds with CI failure on miss -
CoverageThresholds.Evaluatecompares the run's aggregated stats againstcoverage.js.lines,coverage.js.functions, andcoverage.css.rulesfrommotus.config.json(orLaunchOptions.Coverage). When a threshold is set and the run's percentage falls below it, the CLI printsCoverage threshold failed: <metric> <actual>% < <threshold>%and returns a non-zero exit code viaTestRunResult.CoverageThresholdsFailed, so CI fails the build. - Coverage panel in the visual runner - The Blazor visual runner gains a
CoveragePanelalongside the existing timeline and console panels. The panel reads from the newICoverageService/CoverageService, whichTestExecutionServicefeeds with the same snapshots that flow into the per-test sink. Per-test coverage is browsable inline as tests complete; aggregated run-end coverage shows in a summary view. --coverageCLI flag with multi-format output and friendly errors -motus run --coverage console --coverage html:./out --coverage cobertura:./coverage.xmlenables coverage and emits all three formats from a single run. Bare--coveragedefaults to console. The flag flipsMOTUS_COVERAGE_ENABLE=truefor the run so theCoverageCollectorplugin engages without requiring config-file changes. Invalid specs now emit a clean one-line error to stderr instead of dumping a stack trace through the System.CommandLine pipeline:--coverage html(missing target) tells the user the format requires<dir>and gives an example;--coverage coberturadoes the same for<path>;--coverage console:foorejects the spurious target;--coverage xmllists the supported formats;--coverage html:(empty target after the colon) is rejected with the same guidance. CLI returns exit code1for any of these so CI fails fast.--retries Nfor transient CDP disconnects - Re-runs a failing test up toNadditional times when (and only when) the failure chain containsCdpDisconnectedExceptionorMotusTargetClosedException. The check walksInnerException, so wrapped failures still qualify. Non-transient failures (assertion errors, timeouts, real test bugs) are never retried, so flake recovery doesn't mask regressions. Each retry runs the entire test fresh: fresh test instance, freshTestInitialize, fresh browser context, fresh WebSocket. Per-test sink data from a discarded attempt is dropped before the next attempt begins, so coverage/a11y/perf totals don't double-count. A[RETRY] <test> (CDP disconnect, attempt N/total)line is written to stderr for every retry, formatted, indented, and coloured to match the[PASS]/[FAIL]lines so it reads as part of the test stream.- Tracing serialization across browser sessions - CDP
Tracing.startis browser-wide: only one trace can be active per browser process at a time. When multipleBrowserContexts share a browser (e.g. parallel workers under a shared-browser fixture), simultaneousStartAsynccalls used to collide and one would fail with a CDP error.Tracingnow coordinates through aConditionalWeakTable<IMotusSession, SemaphoreSlim>keyed on the browser session, so concurrent starts queue rather than collide. The gate is held untilStopAsync(or untilStartAsyncitself errors out, at which point it rolls back state and releases). StaletracingCompletecontinuations from a previous run are also drained and their pumps cancelled at the top ofStartAsync, preventing one run'sdataCollectedevents from poisoning the next.
Bug Fixes
Profiler.getScriptSourcedoesn't exist; coverage was always 0/0 lines - The original coverage teardown calledProfiler.getScriptSource, which is not a CDP method — script-source retrieval lives on theDebuggerdomain. Every call threw, the catch block logged the error, source stayed empty, andCoverageAggregator.SummarizeScriptreportedTotalLines = 0for every script, collapsing every percentage to0.0%regardless of how much code actually ran. Fixed by enabling theDebuggerdomain alongsideProfilerinOnPageCreatedAsyncand switching the source fetch toDebugger.getScriptSource. A best-effortDebugger.disableruns afterProfiler.stopPreciseCoveragefor symmetry.- Post-disconnect 60-second-per-send hang in
CdpTransport-SendRawAsynconly checked_disposed, not whether the receive loop had already observed a disconnect. After the receive loop exited and faulted in-flight requests, new sends were still accepted, the bytes went out, and each one waited the full 60-secondCommandTimeoutfor a response that could never arrive. With coverage enabled this turned post-disconnect cleanup paths (oneProfiler.takePreciseCoverageplus oneDebugger.getScriptSourceper script plusstopPreciseCoverageplusDebugger.disable) into multi-minute hangs that required Ctrl-C to recover from, masquerading as the test runner having frozen. Fixed by adding a_disconnectedvolatile flag set inside the receive loop's disconnect paths (clean close, exception, and dispose) through a singleMarkDisconnectedAndFaulthelper.SendRawAsyncnow checks_disposed || _disconnectedat the top and throwsCdpDisconnectedExceptionimmediately. A regression test asserts that a send issued after an observed disconnect throws within two seconds rather than the prior sixty.