|
| 1 | +# Predicate Secure Browser Automation Demo |
| 2 | + |
| 3 | +This demo showcases the complete agent loop with: |
| 4 | + |
| 5 | +1. **Pre-execution Authorization** - Policy-based decisions before any browser action |
| 6 | +2. **Browser Automation** - Using PredicateBrowser (@predicatesystems/runtime) with Chrome extension |
| 7 | +3. **Post-execution Verification** - Local LLM (Ollama) or OpenAI for verification planning |
| 8 | + |
| 9 | +## Prerequisites |
| 10 | + |
| 11 | +### 1. Install Dependencies |
| 12 | + |
| 13 | +```bash |
| 14 | +# From the ts-predicate-secure directory |
| 15 | +npm install |
| 16 | + |
| 17 | +# The demo uses @predicatesystems/runtime which includes Playwright |
| 18 | +# Playwright browsers are installed automatically |
| 19 | +``` |
| 20 | + |
| 21 | +### 2. Set Up LLM (Choose One) |
| 22 | + |
| 23 | +#### Option A: Ollama (Local - Recommended) |
| 24 | + |
| 25 | +```bash |
| 26 | +# Install Ollama (macOS) |
| 27 | +brew install ollama |
| 28 | + |
| 29 | +# Start Ollama server |
| 30 | +ollama serve |
| 31 | + |
| 32 | +# Pull the Qwen model (in another terminal) |
| 33 | +ollama pull qwen2.5:7b |
| 34 | +``` |
| 35 | + |
| 36 | +#### Option B: OpenAI (Cloud) |
| 37 | + |
| 38 | +Set your OpenAI API key in `.env`: |
| 39 | + |
| 40 | +```bash |
| 41 | +LLM_PROVIDER=openai |
| 42 | +OPENAI_API_KEY=sk-your-key-here |
| 43 | +LLM_MODEL_NAME=gpt-4o-mini |
| 44 | +``` |
| 45 | + |
| 46 | +### 3. Configure Environment |
| 47 | + |
| 48 | +```bash |
| 49 | +# Copy example environment file |
| 50 | +cp demo/.env.example demo/.env |
| 51 | + |
| 52 | +# Edit as needed |
| 53 | +``` |
| 54 | + |
| 55 | +## Running the Demo |
| 56 | + |
| 57 | +```bash |
| 58 | +# From the ts-predicate-secure directory |
| 59 | +npx ts-node demo/secure-browser-demo.ts |
| 60 | +``` |
| 61 | + |
| 62 | +Or with tsx (faster): |
| 63 | + |
| 64 | +```bash |
| 65 | +npx tsx demo/secure-browser-demo.ts |
| 66 | +``` |
| 67 | + |
| 68 | +## What the Demo Does |
| 69 | + |
| 70 | +1. **Initializes Components** |
| 71 | + - Local LLM verifier (Ollama or OpenAI) |
| 72 | + - SecureAgent with policy file |
| 73 | + |
| 74 | +2. **Starts Browser** |
| 75 | + - Launches Chromium via Playwright |
| 76 | + - Set `BROWSER_HEADLESS=true` for headless mode |
| 77 | + |
| 78 | +3. **Executes Authorized Actions** |
| 79 | + - Navigate to example.com (with authorization check) |
| 80 | + - Take snapshot (verify page loaded) |
| 81 | + - Find and click "More information" link |
| 82 | + |
| 83 | +4. **Verifies Each Action** |
| 84 | + - LLM generates verification predicates |
| 85 | + - Predicates are executed against page state |
| 86 | + - Demo fails if verification fails |
| 87 | + |
| 88 | +## Demo Output |
| 89 | + |
| 90 | +``` |
| 91 | +┌────────────────────────────────────────────────────────────┐ |
| 92 | +│ Predicate Secure Browser Automation Demo │ |
| 93 | +├────────────────────────────────────────────────────────────┤ |
| 94 | +│ Task: Navigate to example.com and verify page loads │ |
| 95 | +│ Start URL: https://www.example.com │ |
| 96 | +│ Principal: agent:demo-browser │ |
| 97 | +└────────────────────────────────────────────────────────────┘ |
| 98 | +
|
| 99 | +Initializing Local LLM Verifier... |
| 100 | +✓ Verifier initialized |
| 101 | +
|
| 102 | +Initializing Secure Agent... |
| 103 | +✓ SecureAgent initialized |
| 104 | + Policy: demo/policies/browser_automation.yaml |
| 105 | + Mode: strict (fail-closed) |
| 106 | + Principal: agent:demo-browser |
| 107 | +
|
| 108 | +Step 1: Initializing Browser... |
| 109 | +✓ Browser started |
| 110 | +
|
| 111 | +Step 2: Executing Browser Task... |
| 112 | +
|
| 113 | +→ Action: navigate (https://www.example.com) |
| 114 | + Pre-execution: Checking authorization... |
| 115 | +✓ Action authorized |
| 116 | + Executing action... |
| 117 | +✓ Action executed |
| 118 | + Post-execution: Generating verification plan... |
| 119 | +[info] Generated 2 verifications |
| 120 | + Reasoning: Verify navigation to example.com succeeded |
| 121 | + Executing verifications... |
| 122 | + [1] url_contains(example.com) |
| 123 | + ✓ Passed |
| 124 | + [2] snapshot_changed() |
| 125 | + ✓ Passed |
| 126 | +✓ All verifications passed |
| 127 | +
|
| 128 | +→ Action: snapshot (current_page) |
| 129 | +... |
| 130 | +
|
| 131 | +✓ Task completed successfully |
| 132 | +
|
| 133 | +┌────────────────────────────────────────────────────────────┐ |
| 134 | +│ Demo completed successfully! │ |
| 135 | +├────────────────────────────────────────────────────────────┤ |
| 136 | +│ Run ID: abc123-def456-... │ |
| 137 | +└────────────────────────────────────────────────────────────┘ |
| 138 | +``` |
| 139 | + |
| 140 | +## Policy File |
| 141 | + |
| 142 | +The demo uses `policies/browser_automation.yaml` which defines: |
| 143 | + |
| 144 | +- **Allowed domains**: example.com, google.com, wikipedia.org |
| 145 | +- **Allowed actions**: navigate, click, snapshot, type |
| 146 | +- **Blocked actions**: Password fields, credit card inputs |
| 147 | +- **Blocked domains**: HTTP (non-HTTPS), malicious sites |
| 148 | + |
| 149 | +## Verification Predicates |
| 150 | + |
| 151 | +The LLM generates verification plans using these predicates: |
| 152 | + |
| 153 | +| Predicate | Description | |
| 154 | +|-----------|-------------| |
| 155 | +| `url_contains(substring)` | Check if URL contains substring | |
| 156 | +| `url_changed` | Check if URL changed after action | |
| 157 | +| `snapshot_changed` | Check if page content changed | |
| 158 | +| `element_exists(selector)` | Check if element exists in DOM | |
| 159 | +| `element_visible(selector)` | Check if element is visible | |
| 160 | +| `text_contains(substring)` | Check if page text contains substring | |
| 161 | + |
| 162 | +## Troubleshooting |
| 163 | + |
| 164 | +### Ollama not available |
| 165 | + |
| 166 | +``` |
| 167 | +Error: Ollama not available at http://localhost:11434 |
| 168 | +``` |
| 169 | + |
| 170 | +Start Ollama: `ollama serve` |
| 171 | + |
| 172 | +### Model not found |
| 173 | + |
| 174 | +``` |
| 175 | +Error: model 'qwen2.5:7b' not found |
| 176 | +``` |
| 177 | + |
| 178 | +Pull the model: `ollama pull qwen2.5:7b` |
| 179 | + |
| 180 | +### Playwright browsers not installed |
| 181 | + |
| 182 | +``` |
| 183 | +Error: Executable doesn't exist |
| 184 | +``` |
| 185 | + |
| 186 | +Install browsers: `npx playwright install chromium` |
| 187 | + |
| 188 | +## Architecture |
| 189 | + |
| 190 | +``` |
| 191 | +┌─────────────────────────────────────────────────────────┐ |
| 192 | +│ SecureBrowserDemo │ |
| 193 | +├─────────────────────────────────────────────────────────┤ |
| 194 | +│ │ |
| 195 | +│ ┌──────────────┐ ┌──────────────┐ ┌────────────┐ │ |
| 196 | +│ │ SecureAgent │ │ Sentience │ │ Verifier │ │ |
| 197 | +│ │ (Policy) │ │ Browser │ │ (LLM) │ │ |
| 198 | +│ └──────────────┘ └──────────────┘ └────────────┘ │ |
| 199 | +│ │ │ │ │ |
| 200 | +│ ▼ ▼ ▼ │ |
| 201 | +│ ┌──────────────────────────────────────────────────┐ │ |
| 202 | +│ │ Authorization Loop │ │ |
| 203 | +│ │ 1. Check policy → 2. Execute → 3. Verify │ │ |
| 204 | +│ └──────────────────────────────────────────────────┘ │ |
| 205 | +│ │ |
| 206 | +└─────────────────────────────────────────────────────────┘ |
| 207 | +``` |
| 208 | + |
| 209 | +## PredicateBrowser |
| 210 | + |
| 211 | +The demo uses `PredicateBrowser` from `@predicatesystems/runtime` which: |
| 212 | + |
| 213 | +- Automatically loads the Sentience Chrome extension |
| 214 | +- Provides `snapshot()` for semantic element detection |
| 215 | +- Supports both free tier (local extension) and API tier (cloud processing) |
| 216 | +- Includes stealth patches to avoid bot detection |
| 217 | + |
| 218 | +## Related |
| 219 | + |
| 220 | +- [predicate-secure SDK](../README.md) - Main SDK documentation |
| 221 | +- [Python Demo](../../py-predicate-secure/demo/) - Python version of this demo |
0 commit comments