Draft
Conversation
feat: add LiveKit benchmark suite with connection performance tests test: add data channel latency benchmarks to measure round-trip times feat: add benchmarks for RPC latency and track publishing performance
refactor: implement Tracer protocol and Span class for structured timing
|
|
…tup/teardown hooks - Collapse InfrastructureMode into cloud(region:) associated value with URL auto-detection - Rename BenchmarkStopwatch to BenchmarkTracer for clarity - Replace custom LockIsolated with SDK's StateSync in EchoParticipant - Rewrite TokenGenerator to use LiveKitUniFFI instead of manual JWT/CommonCrypto - Remove unused TrackPublishBenchmark (requires audio hardware unavailable in CLI) - Extract registerDataChannelBenchmark/registerRpcBenchmark helpers to deduplicate - Use Benchmark setup/teardown hooks so room connections run outside measurement Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- BenchmarkConfig falls back to localhost/devkey/secret when env vars are unset, fixing discovery-phase crashes and removing boilerplate - Remove redundant LIVEKIT_URL/API_KEY/API_SECRET from CI env - Remove push trigger; benchmarks run on schedule and manual dispatch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pwatch API - Rename Span.name → Span.label and add split(label:)/splits compat API - Add Equatable conformance to Span and Span.Entry - Add NoopTracer (default in release) and LoggingTracer (default in debug) - Stopwatch is now a typealias for Span, preserving old public API - Keep connectStopwatch property name to avoid breaking changes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Benchmark-specific microsecond conversion doesn't belong on the core Span type. Moved to a package-local extension in BenchmarkTracer.swift. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Simplify Tracer from a span registry (create/hold/end) to a pure factory (create and forget). Spans now own their lifecycle via an onEnd closure, matching the OpenTelemetry SpanProcessor pattern. - Rename protocol Tracer -> Tracing, file Stopwatch.swift -> Tracing.swift - Add Span.onEnd closure + double-end guard, remove tracer back-reference - Simplify NoopTracer/LoggingTracer to stateless factories - Update BenchmarkTracer to wire onEnd for completed span capture - Deprecate Stopwatch typealias, split(), and splits property - Fix internal usages to use Span type directly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The logging overhead is minimal (one debug log on span end), so a separate no-op implementation is unnecessary. Also flatten BenchmarkTracer's single-field State struct to a direct StateSync. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add benchmark job to CI by calling benchmark.yaml as a reusable workflow. Add workflow_call trigger to benchmark.yaml to support this. TODO: Remove the benchmark job from ci.yaml after the refactor merges. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… payloads Replace 2-size payload testing (100B, 14KB) with 3-tier realistic payloads (200B, 4KB, 15359B) matching actual LiveKit traffic patterns. Expand DC benchmarks to full reliable×lossy matrix and RPC benchmarks to include scaled processing delays. Connection benchmark uses per-instance 25 iterations while DC/RPC use global 100. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ean markdown - Add D_SIGNAL and D_TRANSPORT to connection benchmark metrics config so package-benchmark actually collects and reports them - Split CI into baseline update + baseline read to avoid ANSI progress bars polluting the job summary markdown Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
xianshijing-lk
approved these changes
Mar 23, 2026
| import Foundation | ||
|
|
||
| /// Configuration for benchmark runs, read from environment variables. | ||
| struct BenchmarkConfig { |
Contributor
There was a problem hiding this comment.
do we have similar code for other use cases ? like integration tests ?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.