Skip to content

Commit 1289793

Browse files
committed
web_browsing: register browser surfaces & docs
Convert web_browsing from a window-centric model to a registered browser-surface model and update docs, UI, and tests accordingly. Key changes: - Documentation updated to refer to "registered browser surfaces" (popup windows and inline <x-browser> elements), clarify popup vs inline behavior, and document controls="true" attribute and placement-generic <x-browser> usage. - Added browser-element.js and updated browser-frame-bridge.js, browser-page-content.js, store.js, window.html, window.css and related browser helper scripts to support surface registration, id-based space.browser API, and refined persistence/restore semantics for popup geometry. - Runtime and skill docs adjusted: transient hooks, prompt-time "currently open" and "last interacted" browser sections, and browser-manager/browser-control guidance now reference surfaces and focus/focus-to-front semantics for prompt-time reads. - API / store naming updated (e.g. hasOpenWindows -> hasOpenBrowsers) and agent-facing behavior described as id-based (no per-window handles, no sync), with ref-targeted actions returning { action, state }. - Tests updated and a new server_cluster_read_stress_test.mjs added; browser-related tests adjusted to the new model. These changes enable inline widget-created browser surfaces, unify popup and embedded behaviors, and improve agent/runtime handling of browser content and focus for prompt construction.
1 parent a91a502 commit 1289793

32 files changed

Lines changed: 1845 additions & 314 deletions

File tree

app/AGENTS.md

Lines changed: 3 additions & 3 deletions
Large diffs are not rendered by default.

app/L0/_all/mod/_core/documentation/docs/agent/prompt-and-execution.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ Important details:
4444
- transient runtime context is emitted as its own trailing prepared message when present
4545
- `_core/onscreen_agent` currently adds one short lowercase `chat display mode` transient section only in compact mode so the model sees `chat is in compact mode` and `keep replies short unless more detail is needed for correctness or the user asks for it`; full mode adds no display-mode section
4646
- `_core/onscreen_agent` also appends a bounded `user home files` transient section built from the current user's `~/` tree, omitting `.git/` directories entirely and formatting the remaining paths as a simple indented folder-first listing with `/` suffixes on folders and explicit `# ... more folders` or `# ... more files` summaries when the current defaults `maxDepth: 5`, `maxFoldersPerFolder: 20`, `maxFilesPerFolder: 20`, or `maxLines: 250` hide children
47-
- `_core/web_browsing` may also append a `currently open web browsers` transient section when browser windows exist, using the compact pipe-delimited rows `browser id|url|title` so the agent can see the current browser set without inflating prompt size
48-
- `_core/web_browsing` may also append `last interacted web browser` when a browser targeted by agent-driven open or create or navigation or inspection or interaction helpers plus explicit `state(...)` checks is still open at prompt-build time; that hook fetches fresh simplified `content` for that one browser during transient-section construction instead of caching page content at click or type time, retries a few short settle-and-read passes there so content usually appears without teaching a public `sync(...)` step, and contributes nothing stale when that browser later closes
47+
- `_core/web_browsing` may also append a `currently open web browsers` transient section when registered browser surfaces exist, using the compact pipe-delimited rows `browser id|url|title` so the agent can see the current browser set without inflating prompt size
48+
- `_core/web_browsing` may also append `last interacted web browser` when a browser surface targeted by agent-driven open or create or navigation or inspection or interaction helpers, explicit `state(...)` checks, or direct browser-surface focus is still open at prompt-build time; that hook fetches fresh simplified `content` for that one browser during transient-section construction instead of caching page content at click or type time, retries a few short settle-and-read passes there so content usually appears without teaching a public `sync(...)` step, and contributes nothing stale when that browser later closes
4949
- `_core/promptinclude` may also append a `prompt includes` transient section that lists readable `**/*.transient.include.md` files in alphabetical full-path order and renders each file body in its own fenced block
5050
- `_core/spaces` may append `available spaces` with compact `id|title` rows on any prompt build, and `current space widgets` with compact widget layout rows while a current space is open; post-write `Current Widget` still comes from the runtime transient store
5151

@@ -130,4 +130,4 @@ Important extension families:
130130
- final prompt-input assembly
131131
- execution-plan validation hooks
132132

133-
Current first-party examples include `_core/spaces` for current-space instructions plus the always-on `available spaces` transient section and the in-space `current space widgets` transient section, `_core/promptinclude` for persistent split system/transient include discovery, `_core/memory` for prompt-include-backed `~/memory/` behavior and rolling notes through an auto-loaded system skill, `_core/web_browsing` for the always-loaded onscreen `browser-manager` skill plus brief open-browser status and prompt-time last-interacted browser content in transient context while `browser-control` auto-loads when `browser:open` is present and uses only the top-level numeric-id `space.browser` helpers, and `_core/onscreen_agent` for the compact-mode reply guidance hook plus the bounded current-user `~/` file tree transient section. Module-specific workflow policy still belongs in owner-module skills or owner-module `_core/onscreen_agent/...` JS hooks.
133+
Current first-party examples include `_core/spaces` for current-space instructions plus the always-on `available spaces` transient section and the in-space `current space widgets` transient section, `_core/promptinclude` for persistent split system/transient include discovery, `_core/memory` for prompt-include-backed `~/memory/` behavior and rolling notes through an auto-loaded system skill, `_core/web_browsing` for the always-loaded onscreen `browser-manager` skill plus brief open-browser-surface status and prompt-time last-interacted browser content in transient context while `browser-control` auto-loads when `browser:open` is present and uses only the top-level numeric-id `space.browser` helpers, and `_core/onscreen_agent` for the compact-mode reply guidance hook plus the bounded current-user `~/` file tree transient section. Module-specific workflow policy still belongs in owner-module skills or owner-module `_core/onscreen_agent/...` JS hooks.

app/L0/_all/mod/_core/documentation/docs/agent/skills-and-documentation.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Current first-party context examples:
3636
- `_core/admin/views/shell/shell.html` exports `admin`
3737
- `_core/router/view.html` exports `route:<current-path>`
3838
- `_core/spaces/view.html` exports `space:open` plus `space:id:<id>` when a current space is active
39-
- `_core/web_browsing/window.html` exports `browser:open` while at least one browser window is open
39+
- `_core/web_browsing/window.html` exports `browser:open` while at least one registered browser surface is open, including popup windows and inline `<x-browser>` elements
4040

4141
Examples:
4242

@@ -92,9 +92,9 @@ The first-party `development` tree is intentionally split into narrower nested s
9292

9393
In particular, `development/modules-routing` now teaches custom routed pages as the main alternative to spaces when the user wants a reusable feature surface, shows how to publish dashboard panels through `ext/panels/*.yaml`, and points agents at the importable helper script `/mod/_core/skillset/ext/skills/development/modules-routing/panel-tools.js` instead of pasting one-off browser snippets into the skill text.
9494

95-
The top-level `browser-manager` skill is the short onscreen browser-window guide. It auto-loads on onscreen surfaces, covers opening and closing stand-alone browser windows through the top-level numeric-id `space.browser` helpers, relies on the `currently open web browsers` transient section for the compact `browser id|url|title` readback, and hands in-browser page work off to `browser-control` once a browser window is open.
95+
The top-level `browser-manager` skill is the short onscreen browser-window guide. It auto-loads on onscreen surfaces, covers opening and closing stand-alone browser windows through the top-level numeric-id `space.browser` helpers, relies on the `currently open web browsers` transient section for the compact `browser id|url|title` readback, and hands in-browser page work off to `browser-control` once any browser surface is open.
9696

97-
The top-level `browser-control` skill is the full onscreen browser-automation guide. It auto-loads when the onscreen page also exports `browser:open`, focuses on the top-level numeric-id `space.browser` navigation and inspection helpers plus ref-based page interaction for already-open browser windows, relies on the `currently open web browsers` and `last interacted web browser` transient sections, does not teach per-window handles or an explicit `sync(...)` step, keeps `typeSubmit(...)` defined as type then press Enter in the same field, teaches the lean-token capture rule that `content(...)` uses typed ref boxes such as `[disabled muted button 18]`, `[checked checkbox 7]`, `[image 24]`, or `[input text 30]`, now treats generic event-bound controls as refs too, accepts either `selector` or `selectors` for scoped `dom(...)` and `content(...)` reads, reserves `evaluate(id, script)` or `send(id, "evaluate", { script })` as the last-resort escape hatch when normal helpers are not expressive enough, omits link destinations and label quotes by default, suppresses list bullets while keeping indentation, gives images actionable refs, falls back to truncated URL text for otherwise-empty links or images, and treats those state or semantic tags as best-effort hints that can be suppressed with `includeStateTags: false` or `includeSemanticTags: false` when flatter output matters. The skill also teaches that ref-targeted actions return `{ action, state }` and that `action.status.noObservedEffect` is the main stop-and-reinspect signal for loops, while `detail(id, ref)` remains the preferred way to inspect one link target, image source, DOM target, or control state on demand.
97+
The top-level `browser-control` skill is the full onscreen browser-automation guide. It auto-loads when the onscreen page also exports `browser:open`, focuses on the top-level numeric-id `space.browser` navigation and inspection helpers plus ref-based page interaction for already-open browser surfaces, including widget-authored `<x-browser>` elements, relies on the `currently open web browsers` and `last interacted web browser` transient sections, does not teach per-window handles or an explicit `sync(...)` step, keeps `typeSubmit(...)` defined as type then press Enter in the same field, teaches the lean-token capture rule that `content(...)` uses typed ref boxes such as `[disabled muted button 18]`, `[checked checkbox 7]`, `[image 24]`, or `[input text 30]`, now treats generic event-bound controls as refs too, accepts either `selector` or `selectors` for scoped `dom(...)` and `content(...)` reads, reserves `evaluate(id, script)` or `send(id, "evaluate", { script })` as the last-resort escape hatch when normal helpers are not expressive enough, omits link destinations and label quotes by default, suppresses list bullets while keeping indentation, gives images actionable refs, falls back to truncated URL text for otherwise-empty links or images, and treats those state or semantic tags as best-effort hints that can be suppressed with `includeStateTags: false` or `includeSemanticTags: false` when flatter output matters. The skill also teaches that ref-targeted actions return `{ action, state }` and that `action.status.noObservedEffect` is the main stop-and-reinspect signal for loops, while `detail(id, ref)` remains the preferred way to inspect one link target, image source, DOM target, or control state on demand.
9898

9999
The top-level `spaces` skill is the always-loaded guide for space selection and `space.yaml` work. It stays concise, covers opening and creating and removing spaces plus editing the space file, and relies on the `Available Spaces` transient section for the compact `id|title` readback.
100100

0 commit comments

Comments
 (0)