Commit 72ad5fe
authored
Expose proxy errors as first-class browser telemetry events (#331)
## Summary
The metro egress host-proxy serves branded 5xx error pages carrying an
`X-Kernel-Proxy-Error` header with a typed code when a proxy-layer
failure occurs. This change makes those failures **first-class in
browser telemetry** from the image side: the CDP collector now emits a
dedicated low-volume `proxy_error` event carrying the code, instead of
burying them in the raw `network_response` stream.
## What changed
- **`openapi.yaml` + regenerated `oapi.go` / `category_gen.go`**: new
`BrowserProxyErrorEvent`/`BrowserProxyErrorEventData` schema
(`proxy_error`, category `network`) registered in the
`KnownBrowserTelemetryEvent` discriminated union. `code` is a typed enum
mirroring the proxy's wire values (`destination_blocked`,
`provider_blacklisted`, `provider_unreachable`, `proxy_unavailable`,
`upstream_timeout`, `upstream_dns_failure`, `upstream_connect_failed`);
`status` is a required `int` (502).
- **`cdpmonitor/handlers.go`**: on `Network.responseReceived`, when a
5xx response carries `X-Kernel-Proxy-Error`, emit `proxy_error` with the
header value as `code`, plus `status`, `url`, `method`, `request_id`,
`nav_seq`, and target/frame context. Header lookup is case-insensitive;
origin 5xx pass-through is untouched (no masking). Requests in flight at
CDP attach get their context from the CDP params.
- **Enum validation + rate limit**: `code` is validated against the
generated enum before use, so unknown header values are dropped and the
rate-limit map stays bounded; emission is deduplicated per (session,
code) with a 1s min interval so volume tracks an outage without flooding
the ring.
- **`lib/events/otlpconvert.go`**: `proxy_error` maps to ERROR severity
only for the top-level Document and WARN for subresources; `status`
promotes to `http.response.status_code`.
- **Tests**: classifier/rate-limit unit tests, response-path unit tests
(branded, untracked), and a real-Chromium e2e that serves a branded 502
and asserts the emitted event.
## Notes / trade-offs
- The event is header-driven: it is emitted only when the proxy's
branded page is observed, so the `code` is always the real header value.
No derived/synthetic codes.
- It rides the `network` telemetry category (CDP-derived and opt-in).
Its value is **per-session/per-URL attribution** for sessions already
capturing the network stream — not a default-on alerting signal (proxy
failures are only observable while the CDP collector runs).
- No Chromium patch and no change to the browser fork: the image-side
observation point is the CDP collector. The `oapi.go` diff is large
because it is regenerated (embedded spec + union accessors).
- Deliberately not in scope: making the event default-on (would require
the CDP collector always running), server-side emission from metro, and
a provenance gate (remote addresses can't distinguish proxy-generated
from origin responses; the header is signal-integrity advisory).
## Related
- kernel/kernel #3283 mirrors the schema into the API's
`BrowserTelemetryEvent` union + Stainless SDK models so consumers can
type against `proxy_error`.
## Tests
- `go test ./lib/cdpmonitor/ ./lib/events/` — green (unit +
real-Chromium e2e).
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Adds new telemetry on the hot CDP response path (gated to 502 +
header) and changes OTLP alerting semantics for document vs subresource
proxy failures; scope is bounded by enum validation and sampling.
>
> **Overview**
> Introduces **`proxy_error`** as a first-class network telemetry event
when the CDP monitor classifies a **502** response that carries
**`X-Kernel-Proxy-Error`**, surfacing metro egress proxy failures with a
typed **`code`** instead of only generic `network_response` traffic.
>
> **Schema & pipeline:** OpenAPI adds `BrowserProxyErrorEvent` (enum
codes aligned with metro), regenerated `oapi.go` / `category_gen.go`,
and union wiring. **`cdpmonitor`** detects the header on
`Network.responseReceived` (502-only gate), fills request/nav context
from pending state or CDP params, validates codes against the enum, and
rate-limits to at most one emit per session+code+resource type per
second.
>
> **OTLP:** Promotes `code` to `kernel.proxy_error_code`;
**`proxy_error`** severity is **ERROR** for `Document` and **WARN** for
subresources.
>
> Unit tests, a Chromium e2e, and README taxonomy updates cover the new
path. Events remain opt-in via the network CDP collector category.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
ed5697e. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: chruffins <23645059+chruffins@users.noreply.github.com>1 parent 30573de commit 72ad5fe
12 files changed
Lines changed: 1065 additions & 404 deletions
File tree
- server
- lib
- cdpmonitor
- events
- oapi
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
159 | | - | |
| 159 | + | |
160 | 160 | | |
161 | | - | |
162 | | - | |
| 161 | + | |
| 162 | + | |
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
| |||
171 | 171 | | |
172 | 172 | | |
173 | 173 | | |
174 | | - | |
| 174 | + | |
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
| |||
202 | 202 | | |
203 | 203 | | |
204 | 204 | | |
| 205 | + | |
205 | 206 | | |
206 | 207 | | |
207 | 208 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
7 | 9 | | |
8 | 10 | | |
9 | 11 | | |
| |||
14 | 16 | | |
15 | 17 | | |
16 | 18 | | |
| 19 | + | |
17 | 20 | | |
18 | 21 | | |
19 | 22 | | |
| |||
307 | 310 | | |
308 | 311 | | |
309 | 312 | | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
491 | 491 | | |
492 | 492 | | |
493 | 493 | | |
494 | | - | |
| 494 | + | |
495 | 495 | | |
496 | | - | |
497 | | - | |
498 | | - | |
499 | | - | |
500 | | - | |
501 | | - | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
502 | 507 | | |
503 | 508 | | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
504 | 545 | | |
505 | 546 | | |
506 | 547 | | |
| |||
622 | 663 | | |
623 | 664 | | |
624 | 665 | | |
| 666 | + | |
625 | 667 | | |
626 | 668 | | |
627 | 669 | | |
628 | 670 | | |
629 | 671 | | |
630 | 672 | | |
631 | 673 | | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
632 | 756 | | |
633 | 757 | | |
634 | 758 | | |
| |||
0 commit comments