Skip to content

docs: add cross-origin iframe guidance (CDP WebSocket path) to forge references#15

Open
Wadehl wants to merge 1 commit into
browser-act:mainfrom
Wadehl:feat/iframe-cdp-support
Open

docs: add cross-origin iframe guidance (CDP WebSocket path) to forge references#15
Wadehl wants to merge 1 commit into
browser-act:mainfrom
Wadehl:feat/iframe-cdp-support

Conversation

@Wadehl

@Wadehl Wadehl commented Jul 14, 2026

Copy link
Copy Markdown

Background

During real-world use of browser-act-skill-forge, a gap was encountered: when the target data is rendered inside a cross-origin iframe, the forge references provided no guidance. The eval command is blocked by the browser's same-origin policy, and without a documented fallback, exploration stalls with no clear next step.

Root Cause

The exploration and output references only cover same-origin scenarios. Cross-origin iframes create a hard split in browser-act's command set:

  • state and click work — OS-level accessibility, not subject to same-origin policy
  • eval is blocked at the JS layer

This split was not documented, leaving the forge without a defined path for this case.

Verified Solution

Chrome DevTools Protocol (CDP) exposes every frame — including cross-origin iframes — as an independent debugging target listed at http://localhost:9222/json. By connecting directly to the iframe's webSocketDebuggerUrl via the Python websockets library, arbitrary JS can be executed inside the iframe without touching the browser's same-origin enforcement. This was validated end-to-end: virtual scroll triggering, DOM querying, and structured data extraction all work through this path.

Changes

references/exploration_extraction.md

  • Added a "Cross-Origin iframe" section after DOM Extraction
  • Documents the browser-act capability boundary (state/click ok, eval blocked)
  • Provides the CDP WebSocket direct-connect pattern with Python code examples
  • Defines the decision rule: use state+click when the a11y tree is sufficient; use CDP when JS execution inside the iframe is required
  • Specifies that CDP path scripts are encapsulated as standalone .py files and labeled CDP Direct in the generated Skill

references/output_template.md

  • Added CDP Direct as a new named component type under Capability Components
  • Provides SKILL.md template block for this type (invocation pattern, output example, error handling)
  • Added corresponding Python wrapper template showing the CDP connection pattern (prints result directly, no eval "$(...)" wrapping)
  • Extended Filling Specification #17 to define when the CDP Direct label applies

Design Rationale

The CDP path is Python-native: the WebSocket connection must live in Python, so the script cannot follow the standard "print JS string → eval" contract. Making this a distinct labeled type prevents misuse and keeps the eval "$(...)" wrapper convention unambiguous for all other component types.

…references

When target data is inside a cross-origin iframe, browser-act's eval is blocked
by the browser's same-origin policy. This commit documents the verified fallback:
connect directly to the iframe's CDP WebSocket target, bypassing cross-origin
enforcement at the protocol level.

Changes:
- exploration_extraction.md: add "Cross-Origin iframe" section after DOM Extraction,
  covering browser-act capability boundary (state/click ok, eval blocked), CDP
  WebSocket direct-connect pattern with Python code examples, and decision rule
  for when to use state+click vs CDP
- output_template.md: add "CDP Direct" component type with SKILL.md template block,
  CDP-specific Python wrapper template (connects to CDP and prints result directly,
  no eval wrapping), and extend Filling Specification #17 to include the new label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant