Repository navigation
Week 3: Python adapters + S3-S5 benchmarks + MCP source adapter - #3
Merged
Merged
Conversation
Adds Phase 4 / Week 3: Python adapters (adapters/python/, Apache 2.0): - acp_common: shared core (Manifest dataclasses, ACPClient over urllib, schema translation, topological_order with cycle detection, upstream_url_for helper). - acp_openai: manifest_to_openai_tools() + dispatch_tool_call() that routes through the auth-injection proxy. Accepts both dict and OpenAI SDK object shapes for tool calls. - acp_langgraph: make_node() + build_graph() produce a StateGraph with topological edges. Falls back gracefully when langgraph is missing. - acp_crewai: make_tool_callable() + manifest_to_crewai_tools() produce CrewAI BaseTool instances (or plain callables when crewai is missing). - 23 pytest cases covering all four packages with mocked urlopen transports. Benchmark expansion: - Scenario gained mcp_noise_tools field. - mcp_context_payload(noise_tools=N) spreads generic realistic descriptors across MCP servers to model 'many-registered-few-relevant'. - Harness now ships S1, S2, S3 (DAG), S4 (scale, 50 tools), S5 (auth-heavy). MCP source adapter (internal/sources/mcp, BSL 1.1): - Importer.ImportSource() GETs <base>/tools/list and registers every returned descriptor as an ACP registry.Tool. - Capability inference from tool name plus verb synonyms. - Schema compaction from JSON-Schema -> ACP mini-language (string, int?, string[], enum:a|b|c). This is what produces the token reduction on the ACP side. - Consumer-defined HTTPDoer interface; mocked with go.uber.org/mock. - 7 test cases (89.4% line coverage on the new package). First measured S1-S5 numbers (50 runs/scenario, tiktoken cl100k_base): S1 Simple DB query ACP 111 / MCP 373 -> 70.2% S2 Multi-tool workflow ACP 295 / MCP 837 -> 64.7% S3 Complex DAG ACP 306 / MCP 1257 -> 75.6% S4 Scale (50 tools, 2 rel) ACP 241 / MCP 9223 -> 97.4% S5 Auth-heavy ACP 359 / MCP 1431 -> 74.9% S4 is the headline: at 50 registered tools with only 2 relevant to the task, ACP delivers a 97.4% reduction over MCP's full tools/list payload. Round-trips before first useful action drop from 3-21 (MCP) to 1 (ACP). Raw + summary committed at: - results/2026-05-02-week3-baseline.json (250 raw runs) - results/2026-05-02-week3-summary.md Validation: - go vet, go test -race -count=1 ./... (9 packages green) - staticcheck, govulncheck both clean (zero vulnerabilities) - 35 pytest cases pass (12 benchmark + 23 adapter) - Coverage 89.6% (Go statements) Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 4 / Week 3 of the ACP PoC. Three deliverables in one PR:
`acp_crewai`) — ACP is now usable from the Python agent ecosystem.
knob that models intent-scoping at scale.
upstream MCP server's `tools/list` and registers every tool in
the ACP registry. Concrete proof of "ACP on top of MCP".
Headline benchmark numbers (S1-S5)
50 runs per scenario, `tiktoken/cl100k_base`, live ACP server:
S4 is the pitch-deck headline: at 50 registered tools (only 2
relevant), ACP delivers a 97.4% token reduction over MCP's full
`tools/list` payload. Round-trips before the first useful action drop
from 3-21 (MCP) to 1 (ACP).
Adapter design
Common pattern across all three:
ACPClient over urllib, schema translation, topological ordering).
tools out — and routes execution through the auth-injection proxy
so credentials never enter the agent context window.
langgraph/crewai/openai (the adapters work with them when present;
they fall back to plain callables when not).
MCP source adapter
`Importer.ImportSource(Source)` does a `GET /tools/list` against an
upstream MCP server, then for each descriptor:
(`string`, `int?`, `string[]`, `enum:a|b|c`).
the egress allow-list entry.
This is what makes the "MCP ecosystem becomes our supply chain" claim
real: point ACP at an existing MCP server and it produces 65-97%
smaller manifests for the same task.
Tests added
acp_crewai).
boundary, coverage 89.4%).
Validation
```
go vet ./... # clean
go test -race -count=1 ./... # 9 pkg green
staticcheck ./... # clean
govulncheck ./... # No vulnerabilities found
pytest benchmark/tests adapters/python/tests # 35 passed
coverage # 89.6% Go statements
```
Verifier
Independent verifier subagent ran before commit: PASS. The two
non-blocking polish items (license docstrings on test files,
explicit staging of result files) were addressed where they
mattered.