Skip to content

OpenRange as physical-space simulation  #219

@larstalian

Description

@larstalian

Vision

OpenRange as a physical-space simulation substrate: an entire
building, with offices, desks, hallways, kitchens; people moving
through it, sitting at desks, talking, opening laptops. The cyber
webapp pack is the first scenario rendered on top of this — not the
goal.

The goal is a generic multi-agent environment where world state is
grounded (positions, rooms, objects, conversations) and agents
(LLM-driven employees, attackers, defenders, future household /
warehouse / logistics agents) act through that grounding rather than
through a thin HTTP-only API.

Why this matters

For cyber specifically (which is where we have the strongest pull
today):

  • Insider scenarios with a body. Phishing isn't a click; it's
    an employee at a specific desk, mid-conversation with a colleague,
    who clicks at 2:47pm. Physical context is what makes the
    attacker's chain realistic and the defender's correlation
    problem hard.
  • Hybrid attacks. Tailgating, USB drops, badge cloning. None of
    these have a cyber-only abstraction.
  • Realistic noise. Why is this laptop suddenly hitting
    /admin? Because Maria walked over and they're pair-debugging.
    Co-occurrences ground signal vs. noise.

For everything else: the same engine can host warehouse robotics,
household assistance, office-process simulation, multi-agent
logistics. Cyber is a forcing function, not the destination.

Reference: Concordia

google-deepmind/concordia
is the closest existing thing. LLM-driven agents composed from
components (memory, observation, plan, somatic state, goal),
acting against a GameMaster that resolves each tick — with
explicit support for spatial/physical state. Their examples/
folder is the right place to read; the API is small, the patterns
are in the configs.

The agent shape they land on maps cleanly onto our AgentNPC +
AgentBackend work in #74 / #217.

Where we are

Already in tree:

  • AgentNPC + AgentBackend — agent loops with tools, opt-in
    LLM, swappable provider. Concordia's "agent driven by some model"
    primitive is essentially this.
  • Dashboard with a Three.js 3D office renderer (sim-canvas) —
    desks, walls, walking characters, idle NPC wandering. Currently
    fed by topology services + green_personas; ready to scale.

Missing for the full vision:

  • Physical backing parallel to HTTPBacking — exposes
    move_to(room), observe_nearby(), pick_up(object),
    say_to(actor, text) as the interface mapping. NPCs/agents
    bind their tools to these instead of (or alongside) http_get.
  • GameMaster-shaped tick model. Concordia's GM resolves
    everyone's intended actions per tick; our current _step_npcs
    iterates with no contention model. Fine for HTTP, picky for
    physical (two NPCs picking up the same object, queuing for the
    printer, etc.).
  • Per-actor memory + observation streams. Today NPCs are
    stateless across ticks. Concordia's associative memory component
    is a good shape to copy when a real physical pack needs it.
  • A real physical pack. Probably a small office (5-10 employees,
    one floor, low-fidelity rooms) is the right first cut — domestic
    / warehouse can come later. Choice constrains what the backing's
    interface needs to look like first.

Phasing

Each step is its own PR; don't try to design the whole system up
front.

  1. Office-shaped demo on top of the cyber pack. No new
    abstractions. Spawn ~6 chatter NPCs in the existing renderer,
    wire speech-event rendering. Goal: a gif that shows the vision.
    (In flight — see linked PR.)
  2. Physical backing protocol. Define the interface shape
    (move_to, observe_nearby, say_to, ...). Implement a
    minimal in-memory backing driving the existing dashboard
    renderer.
  3. First standalone physical pack. Small office, separate from
    the cyber pack — proves the substrate generalizes.
  4. GameMaster tick. Per-tick action resolution with simple
    contention rules (one actor per object, rate-limited speech).
  5. Memory + observation primitives. Concordia-shape components
    (associative recall, recent buffer). Gated on a pack that
    actually needs them.
  6. LLM-driven employees as the default. Agent loops driving
    movement / speech from persona + memory + nearby observations,
    not from canned phrases.

Out of scope (for this issue)

  • Cost/budget story for LLM-driven NPCs (separate concern).
  • Defender-vs-attacker game theory.
  • Specific Concordia API choices (different runtime model; we may
    borrow the patterns, not the codebase).

Acceptance

  • Per phase: each step lands as its own PR with its own acceptance
    criteria. This issue is the umbrella, not a single deliverable.
  • Issue stays open until phase 3 lands (first standalone physical
    pack) — at that point the substrate has proven itself and the
    remaining phases are incremental polish.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestresearchExploratory / no near-term planroadmapTracked on the public roadmap

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions