ci: add GitHub Actions workflow and clean up obsolete smoke script; update docs

Author: zelda2003

.github/workflows/ci.yml

@@ -0,0 +1,36 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  build-test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Rust
+        uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+          components: rustfmt, clippy
+
+      - name: Build release MCP binary
+        run: cargo build --release --all-features --bin cortex-scout-mcp
+
+      - name: Run Rust unit tests
+        run: cargo test --all-features -- --quiet
+
+      - name: Install Python dependencies
+        run: |
+          python3 -m pip install --upgrade pip
+
+      - name: Run repo validation script
+        run: python3 publish/ci/validate.py
+
+      - name: Run MCP smoke test
+        run: python3 publish/ci/smoke_mcp.py

.vscode/mcp.json

@@ -0,0 +1,19 @@
+{
+  "servers": {
+    "cortex-scout": {
+      "type": "stdio",
+      "command": "env",
+      "args": [
+        "RUST_LOG=warn",
+        "SEARCH_ENGINES=google,bing,duckduckgo,brave",
+        "LANCEDB_URI=/Users/hero/Documents/GitHub/ShadowCrawl/lancedb",
+        "HTTP_TIMEOUT_SECS=30",
+        "HTTP_CONNECT_TIMEOUT_SECS=10",
+        "MAX_CONTENT_CHARS=10000",
+        "IP_LIST_PATH=/Users/hero/Documents/GitHub/ShadowCrawl/ip.txt",
+        "PROXY_SOURCE_PATH=/Users/hero/Documents/GitHub/ShadowCrawl/proxy_source.json",
+        "/Users/hero/Documents/GitHub/ShadowCrawl/mcp-server/target/release/cortex-scout-mcp"
+      ]
+    }
+  }
+}

README.md

@@ -90,21 +90,26 @@ Basic build (search, scrape, deep research, memory):
 ```bash
 git clone https://github.com/cortex-works/cortex-scout.git
 cd cortex-scout/mcp-server
-cargo build --release
+cargo build --release --bin cortex-scout-mcp
 ```
 
 Full build (includes `hitl_web_fetch` / visible-browser HITL):
 
 ```bash
-cargo build --release --all-features
+cargo build --release --all-features --bin cortex-scout-mcp
 ```
 
----
+If you also want the optional HTTP server binary, build it explicitly with `cargo build --release --bin cortex-scout`.
 
-## MCP Integration (VS Code / Cursor / Claude Desktop)
+Local MCP smoke test:
 
-Add a server entry to your MCP config.
+```bash
+python3 publish/ci/smoke_mcp.py
+```
 
+This runs a newline-delimited JSON-RPC stdio session against the local `cortex-scout-mcp` binary and exercises the main public tools with safe example inputs.
+ 
+A GitHub Actions workflow (`.github/workflows/ci.yml`) invokes the same validation, Rust test suite, and the `smoke_mcp.py` smoke test on every push/PR. It replaces the older HTTP-mode smoke script (`publish/ci/smoke.py`), which has been removed as obsolete.
 **VS Code** (`mcp.json` — global, or `settings.json` under `mcp.servers`):
 
 ```jsonc
@@ -121,16 +126,15 @@ Add a server entry to your MCP config.
         "LANCEDB_URI=/YOUR_PATH/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/YOUR_PATH/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/YOUR_PATH/cortex-scout/proxy_source.json",
-        "--",
         "/YOUR_PATH/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
   }
 }
 ```
 
+Default behavior is direct/no-proxy. Add `IP_LIST_PATH` and `PROXY_SOURCE_PATH` only if you want proxy tools available. If you want `proxy_control` available without routing normal traffic through proxies, point `IP_LIST_PATH` at an empty `ip.txt` file and let agents populate it on demand.
+
 > **Important:** Always use `RUST_LOG=warn`, not `info`. At `info` level, the server emits hundreds of log lines per request to stderr, which can confuse MCP clients that monitor stderr.
 
 > **Windows:** Windows has no `env` command. Use the `command`+`env` object format instead — see [docs/IDE_SETUP.md](docs/IDE_SETUP.md).
@@ -149,15 +153,12 @@ Add a server entry to your MCP config.
         "LANCEDB_URI=/YOUR_PATH/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/YOUR_PATH/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/YOUR_PATH/cortex-scout/proxy_source.json",
         "OPENAI_BASE_URL=https://openrouter.ai/api/v1",
         "OPENAI_API_KEY=sk-or-v1-...",
         "DEEP_RESEARCH_LLM_MODEL=moonshotai/kimi-k2.5",
         "DEEP_RESEARCH_ENABLED=1",
         "DEEP_RESEARCH_SYNTHESIS=1",
         "DEEP_RESEARCH_SYNTHESIS_MAX_TOKENS=4096",
-        "--",
         "/YOUR_PATH/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
@@ -222,8 +223,8 @@ Create `cortex-scout.json` in the same directory as the binary (or repository ro
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `IP_LIST_PATH` | — | Path to `ip.txt` (one proxy per line: `http://`, `socks5://`) |
-| `PROXY_SOURCE_PATH` | — | Path to `proxy_source.json` (used by `proxy_control grab`) |
+| `IP_LIST_PATH` | — | Optional path to `ip.txt` (one proxy per line: `http://`, `socks5://`). Leave unset to disable proxy support entirely, or point at an empty file to keep proxy tools available but inactive by default |
+| `PROXY_SOURCE_PATH` | — | Optional path to `proxy_source.json` (used by `proxy_control grab`) |
 
 ### Semantic Memory (LanceDB)
 
@@ -309,7 +310,7 @@ Prevention checklist:
 Check these before anything else:
 
 1. Use `RUST_LOG=warn`, not `info`.
-2. On macOS/Linux `env`-style configs, include `"--"` before the binary path.
+2. On macOS/Linux `env`-style configs, pass the binary path directly after the env assignments. Do not insert `"--"` in `mcp.json` args.
 3. On Windows, do not use `env`; use `command` plus an `env` object.
 4. Make sure the binary path points to a current build, not an old pre-fix binary.
 

docs/IDE_SETUP.md

@@ -58,16 +58,15 @@ VS Code reads MCP servers from two places (both are valid):
         "LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json",
-        "--",
         "/absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
   }
 }
 ```
 
+Default behavior is direct/no-proxy. Add `IP_LIST_PATH` and `PROXY_SOURCE_PATH` only if you want proxy support available. For opt-in proxy usage, keep `ip.txt` empty and let the agent call `proxy_control grab` before retrying with `use_proxy: true`.
+
 ### Windows
 
 Windows has no `env` command. Use the `command` + `env` object format:
@@ -85,9 +84,7 @@ Windows has no `env` command. Use the `command` + `env` object format:
         "SEARCH_ENGINES": "google,bing,duckduckgo,brave",
         "LANCEDB_URI": "C:\\Users\\YOU\\cortex-scout\\lancedb",
         "HTTP_TIMEOUT_SECS": "30",
-        "MAX_CONTENT_CHARS": "10000",
-        "IP_LIST_PATH": "C:\\Users\\YOU\\cortex-scout\\ip.txt",
-        "PROXY_SOURCE_PATH": "C:\\Users\\YOU\\cortex-scout\\proxy_source.json"
+        "MAX_CONTENT_CHARS": "10000"
       }
     }
   }
@@ -117,9 +114,6 @@ Claude uses `"mcpServers"` (not `"servers"`).
         "LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json",
-        "--",
         "/absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
@@ -147,9 +141,6 @@ Cursor stores MCP config in `~/.cursor/mcp.json` (also configurable via UI).
         "LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json",
-        "--",
         "/absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
@@ -176,9 +167,6 @@ Windsurf config file: `~/.codeium/windsurf/mcp_config.json`
         "LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json",
-        "--",
         "/absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
@@ -208,9 +196,6 @@ mcpServers:
       - LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb
       - HTTP_TIMEOUT_SECS=30
       - MAX_CONTENT_CHARS=10000
-      - IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt
-      - PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json
-      - --
       - /absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp
 ```
 
@@ -239,6 +224,6 @@ These require an `OPENAI_API_KEY` (or compatible endpoint) in the environment.
 |---------|-----|
 | "no tools found" | Check binary path is correct and executable (`chmod +x`) |
 | Tools time out immediately | Ensure `RUST_LOG=warn` — not `info` or `debug` |
-| Proxy tools fail | Confirm `ip.txt` and `proxy_source.json` exist and path vars are set |
+| Proxy tools fail | Proxy support is optional. If you want it, set `IP_LIST_PATH`/`PROXY_SOURCE_PATH`; keep `ip.txt` empty by default and populate it only when needed |
 | `hitl_web_fetch` not available | Binary must be built with `--all-features` |
 | Config not picked up | Fully restart the client app after editing the JSON/YAML |

docs/VSCODE_SETUP.md

@@ -56,16 +56,15 @@ VS Code reads MCP servers from two places:
         "LANCEDB_URI=/absolute/path/to/cortex-scout/lancedb",
         "HTTP_TIMEOUT_SECS=30",
         "MAX_CONTENT_CHARS=10000",
-        "IP_LIST_PATH=/absolute/path/to/cortex-scout/ip.txt",
-        "PROXY_SOURCE_PATH=/absolute/path/to/cortex-scout/proxy_source.json",
-        "--",
         "/absolute/path/to/cortex-scout/mcp-server/target/release/cortex-scout-mcp"
       ]
     }
   }
 }
 ```
 
+Default behavior is direct/no-proxy. Add `IP_LIST_PATH` and `PROXY_SOURCE_PATH` only if you want proxy support available. For an opt-in proxy setup, keep `ip.txt` empty and let `proxy_control grab` populate it only when an agent decides a retry should use proxies.
+
 ### Windows
 
 Windows has no `env` command. Pass env vars as an object instead:
@@ -83,9 +82,7 @@ Windows has no `env` command. Pass env vars as an object instead:
         "SEARCH_ENGINES": "google,bing,duckduckgo,brave",
         "LANCEDB_URI": "C:\\Users\\YOU\\cortex-scout\\lancedb",
         "HTTP_TIMEOUT_SECS": "30",
-        "MAX_CONTENT_CHARS": "10000",
-        "IP_LIST_PATH": "C:\\Users\\YOU\\cortex-scout\\ip.txt",
-        "PROXY_SOURCE_PATH": "C:\\Users\\YOU\\cortex-scout\\proxy_source.json"
+        "MAX_CONTENT_CHARS": "10000"
       }
     }
   }
@@ -126,4 +123,4 @@ After editing any MCP config, **restart VS Code** (Cmd+Q / Alt+F4) to reload the
 | Tools time out | Set `RUST_LOG=warn` (not `info` or `debug`) |
 | `hitl_web_fetch` missing | Rebuild with `--all-features` |
 | Config not picked up | Fully restart VS Code after editing mcp.json |
-| Proxy tools fail | Confirm `ip.txt` and `proxy_source.json` exist and paths are set |
+| Proxy tools fail | Proxy support is optional. If you want it, set `IP_LIST_PATH`/`PROXY_SOURCE_PATH`; keep `ip.txt` empty by default and populate it only when needed |