UnrealCV Runtime MCP

Public client examples and agent skills for the Runtime MCP service in UnrealCV Dev For UnrealZoo.

The Runtime MCP server is currently distributed with supported UnrealZoo environments and is tested there first. This repository does not contain the server's Unreal Engine C++ implementation.

Examples

Complex Scene Navigation

The agent spawns a character, switches to first-person view, finds two sakura trees, approaches the second tree to within 0.5 meters, and captures a final third-person composition.

1. Initial First Person View	2. First Sakura Reached	3. Second Sakura Reached
Try to find the sakura tree	The character navigates to the first sakura	The character approaches the second sakura to within 0.5 m

4. Front Framing	5. Left Framing	6. Right Framing	7. Final Capture
Camera detached and moved to the front	Composition evaluated from the left	Composition evaluated from the right	Character and sakura framed together

Prompt: View the original prompt · Workflow: View the execution details

Scene Captioning

The agent uses UnrealCV Runtime MCP tools to perceive the scene, rotate the camera, and capture views in six directions. These information are then used to produce a caption of the complete environment.

The six images below are the resulting north, east, south, west, upward, and downward captures produced by UnrealCV Runtime MCP.

North	East	South

West	Up	Down

A vibrant, compact stylized Tokyo district of dense mid-rise buildings, neon signage, elevated structures, narrow streets and sidewalks, all threaded with abundant pink cherry blossoms beneath a bright blue sky.

Prompt: View the original prompt · Workflow: View the captioning process · Result: View the caption

Blueprint Function Calling (Change Character Appearance)

The agent uses UnrealCV Runtime MCP tools to discover the character's Blueprint API, call set_app(NewParam) to switch its appearance, and photograph each result.

The ten images below were captured by the agent after applying the ten appearance variants, using UnrealCV Runtime MCP capture tools.

Prompt: View the original prompt · Workflow: View the execution details

Scene Generation In Action

Given a natural-language brief and a world coordinate, an agent can raycast to the ground, inspect native scene state, search assets and bounds, spawn and settle assets without overlaps, then return an auditable six-view result. This is a real Runtime MCP run in the Tokyo environment: a bench, table, and traffic cone were added to a street-side rest point and validated before capture.

The scene-generation video shows the same workflow as a runtime sequence: the agent creates the scene, adds a character asset, and moves the character to the generated bench in response to a natural-language instruction.

Top-down evaluation	X+ diagonal evaluation	X- diagonal evaluation
Y+ diagonal evaluation	Y- diagonal evaluation

The full tool audit, asset paths, bounds, ground hit, placement validation, and capture provenance are recorded in examples/scene_generation_demo/manifest.json. Captures use MQRC at 640x360 by default, capped at 1280x720, so visual checks remain useful without needlessly expanding an agent's image context.

Get Started

Universal configuration

Add the following configuration to a local .mcp.json file or to your coding agent's MCP configuration file:

{
  "mcpServers": {
    "unrealcv": {
      "type": "http",
      "url": "http://127.0.0.1:29998/mcp",
      "disabled": false
    }
  }
}

Codex configuration

Add the following configuration to ~/.codex/config.toml:

[mcp_servers.unrealcv]
url = "http://127.0.0.1:29998/mcp"
enabled = true

Supported Protocols

Layer	Current Support
Application Protocol	Model Context Protocol (MCP)
MCP Versions	2025-11-25, 2025-06-18, 2025-03-26, 2024-11-05
RPC	JSON-RPC 2.0
Transport	Streamable HTTP
Standard Responses	application/json
Streaming Responses	text/event-stream using SSE message events
Session Management	Mcp-Session-Id
Protocol Version Header	Mcp-Protocol-Version
Default Endpoint	http://127.0.0.1:29998/mcp

Availability

• Open-source UnrealCV commands: https://docs.unrealcv.org/en/latest/reference/commands.html
• UnrealCV Dev For UnrealZoo documentation: https://docs.unrealcv.org/en/latest/unrealcv_plus/index.html
• UnrealZoo environments: https://unrealzoo.github.io/

License

MIT. See LICENSE.