Skip to content

Latest commit

 

History

History
2086 lines (1607 loc) · 98 KB

File metadata and controls

2086 lines (1607 loc) · 98 KB

Morphic-Agent Implementation Plan

Master implementation plan — 7 phases, 14 weeks Phase 1 has sprint-level detail. Phases 2-7 have week-level plans.


Design Principles (All Phases)

1. Prove $0 operation first (LOCAL_FIRST)
2. Default independent tasks to parallel (DEFAULT TO PARALLEL)
3. Embed Context Engineering (Manus 5 principles) from Phase 1
4. Guarantee behavior with E2E tests at each phase end
5. "If you declare it, execute it" — no stubs without follow-through
6. OSS-First: use established libraries, custom code only for domain logic
7. CLI + API: both are first-class interfaces calling the same use cases

Dependency Graph

[Infrastructure]
  PostgreSQL+pgvector, Redis, Neo4j, Docker Compose
       │
       ▼
[LLM Layer]  ←── Phase 1 top priority
  Ollama Manager → LiteLLM Router → Cost Tracker
       │
       ▼
[Task Graph Engine]  ←── Phase 1 core
  LangGraph DAG → Scheduler → Parallel Execution
       │
       ├──────────────────────┐
       ▼                      ▼
[Context Engineering]    [Semantic Memory]
  KV-Cache Optimizer      mem0 + pgvector
  Tool State Machine      Neo4j Knowledge Graph
  todo.md Manager         Context Zipper (simple)
  Observation Diversifier
       │                      │
       └──────────┬───────────┘
                  ▼
[API + CLI Layer]  FastAPI + WebSocket + typer
                  │
                  ▼
[UI Layer]   Next.js 15 + Shadcn/ui
                  │
                  ▼
         *** Phase 1 Complete ***
                  │
       ┌──────────┼──────────┐
       ▼          ▼          ▼
[Phase 2]    [Phase 3]   [Phase 4]
Parallel &   Memory &    Agent CLI
Planning     Context     Orchestration
+ CLI v1
       │          │          │
       └──────────┼──────────┘
                  ▼
       ┌──────────┼──────────┐
       ▼          ▼          ▼
[Phase 5]    [Phase 6]   [Phase 7]
Marketplace  Evolution   A2A & Scale

Phase 1: Foundation (Week 1-2) ✅ COMPLETE

Goal: Complete a minimal agent loop that runs at $0 with Ollama Deliverable: User inputs a goal → DAG generated → Ollama executes → results displayed + cost $0

Result: 7/7 sprints complete. 298 unit tests + 26 integration tests, all pass. Full stack operational (API + UI + WebSocket).

Sprint 1.1: Infrastructure (Day 1-2) — COMPLETE

Goal: All infrastructure starts with docker compose up -d, DB schema complete

Files to Create

pyproject.toml                          # uv project definition
docker-compose.yml                      # PostgreSQL+pgvector, Redis, Neo4j
.env.example                            # Environment variable template
alembic.ini                             # DB migration config
domain/                                 # Clean Architecture Layer 1
infrastructure/persistence/database.py  # SQLAlchemy async engine + pgvector
infrastructure/persistence/models.py    # ORM models
shared/config.py                        # pydantic-settings config
migrations/env.py                       # Alembic async environment

Docker Compose Services

services:
  postgres:
    image: pgvector/pgvector:pg16        # pgvector extension pre-installed
    ports: ["5432:5432"]
    volumes: [pgdata:/var/lib/postgresql/data]

  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]
    # Queue only. No persistence needed (Phase 1)

  neo4j:
    image: neo4j:5-community
    ports: ["7474:7474", "7687:7687"]    # Browser + Bolt
    volumes: [neo4jdata:/data]

DB Schema (PostgreSQL)

CREATE TABLE tasks (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    goal TEXT NOT NULL,
    status VARCHAR(20) DEFAULT 'pending',
    parent_id UUID REFERENCES tasks(id),
    depth INT DEFAULT 0,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT now(),
    updated_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE task_executions (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    task_id UUID REFERENCES tasks(id) NOT NULL,
    model_used VARCHAR(100) NOT NULL,
    prompt_tokens INT,
    completion_tokens INT,
    cost_usd DECIMAL(10,6) DEFAULT 0,
    latency_ms INT,
    result TEXT,
    error TEXT,
    cache_hit BOOLEAN DEFAULT false,
    created_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE memories (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    content TEXT NOT NULL,
    embedding vector(1536),               -- pgvector
    memory_type VARCHAR(20) NOT NULL,
    access_count INT DEFAULT 1,
    importance_score FLOAT DEFAULT 0.5,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT now(),
    last_accessed TIMESTAMPTZ DEFAULT now()
);

CREATE INDEX ON memories
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

CREATE TABLE cost_logs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    model VARCHAR(100) NOT NULL,
    prompt_tokens INT DEFAULT 0,
    completion_tokens INT DEFAULT 0,
    cost_usd DECIMAL(10,6) DEFAULT 0,
    cached_tokens INT DEFAULT 0,
    is_local BOOLEAN DEFAULT false,
    created_at TIMESTAMPTZ DEFAULT now()
);

Neo4j Schema (L3 Knowledge Graph)

CREATE CONSTRAINT entity_name IF NOT EXISTS
  FOR (e:Entity) REQUIRE e.name IS UNIQUE;
-- Phase 1 uses dynamic labels for flexibility

Completion Criteria

  • docker compose up -d starts all 3 services
  • alembic upgrade head creates schema successfully
  • pgvector vector type insert + search test passes
  • Neo4j Cypher query node create + search test passes
  • shared/config.py loads .env + pydantic validation

Sprint 1.2: LLM Layer (Day 3-4) — COMPLETE

Goal: Local inference with Ollama at $0, cost tracking functional

Files to Create

infrastructure/llm/__init__.py
infrastructure/llm/router.py              # MultiLLMRouter (LiteLLM integration)
infrastructure/llm/ollama_manager.py       # OllamaManager
infrastructure/llm/cost_tracker.py         # CostTracker (callback-based)
tests/unit/infrastructure/test_litellm_gateway.py
tests/unit/infrastructure/test_ollama_manager.py

OllamaManager Spec

class OllamaManager:
    """Ollama lifecycle management"""
    base_url: str = "http://127.0.0.1:11434"

    async def is_running(self) -> bool:
        """Health check via GET /api/tags"""

    async def list_models(self) -> list[str]:
        """List installed models"""

    async def pull_model(self, model: str) -> None:
        """Pull model via POST /api/pull"""

    async def ensure_model(self, model: str) -> bool:
        """Pull if missing, return True if available"""

    def get_recommended_model(self, ram_gb: int) -> str:
        """Recommend model based on machine specs
        8GB  → qwen3:8b
        16GB → qwen3:8b (default)
        32GB → qwen3-coder:30b
        GPU  → llama3.3:70b
        """

MultiLLMRouter Spec

class MultiLLMRouter:
    MODEL_TIERS = {
        "free":   ["ollama/qwen3-coder:30b", "ollama/qwen3:8b", ...],
        "low":    ["claude-haiku-4-5-20251001", "gemini/gemini-2.0-flash"],
        "medium": ["claude-sonnet-4-6", "gpt-4o-mini", ...],
        "high":   ["claude-opus-4-6", "gpt-4o"],
    }

    async def route(self, task_type: str, budget_remaining: float) -> str:
        """Select optimal model from task type + remaining budget
        1. LOCAL_FIRST: prefer free tier if Ollama is running
        2. Budget check: force free tier if budget exhausted
        3. Task type → tier → first available model
        """

    async def call(self, model: str, messages: list, **kwargs) -> LLMResponse:
        """LiteLLM completion() wrapper
        cache={"type": "disk"} for disk caching
        Swap api_base for Ollama
        """

CostTracker Spec

class CostTracker:
    """Real-time cost tracking via LiteLLM success_callback"""

    async def on_success(self, kwargs, response, start_time, end_time):
        """LiteLLM callback: record cost of every LLM call to DB"""

    async def get_daily_total(self) -> float:
    async def get_monthly_total(self) -> float:
    async def get_local_usage_rate(self) -> float:
    def check_budget(self, budget_usd: float) -> bool:
    async def get_savings_from_local(self) -> float:

Completion Criteria

  • Ollama inference with qwen3-coder:30b → response received (OllamaManager + LiteLLMGateway)
  • LiteLLM → Ollama call → recorded in cost_logs (cost=0) (CostTracker.record)
  • With API key: Claude Haiku call → recorded in cost_logs (cost>0) (CostTracker tests)
  • get_local_usage_rate() returns accurate ratio (14 tests)
  • Router forces free tier when budget exhausted (7 routing tests)
  • qwen3 thinking mode disabled via extra_body={'think': False} (TD-015)
  • is_available() verifies model installed in Ollama (TD-014)

Sprint 1.3: Task Graph Engine (Day 5-7) — COMPLETE

Goal: Goal input → LLM decomposition → DAG generation → execution → result

Files Created

domain/ports/task_engine.py                # TaskEngine ABC (decompose + execute)
application/use_cases/create_task.py       # CreateTaskUseCase (decompose → persist)
application/use_cases/execute_task.py      # ExecuteTaskUseCase (DAG → status → persist)
infrastructure/task_graph/__init__.py      # Re-exports
infrastructure/task_graph/state.py         # AgentState TypedDict
infrastructure/task_graph/intent_analyzer.py # IntentAnalyzer (LLM goal → subtasks)
infrastructure/task_graph/engine.py        # LangGraphTaskEngine (DAG + parallel + retry)
tests/unit/application/__init__.py
tests/unit/application/test_create_task.py      # 5 tests
tests/unit/application/test_execute_task.py     # 6 tests
tests/unit/infrastructure/test_intent_analyzer.py    # 6 tests
tests/unit/infrastructure/test_task_graph_engine.py  # 9 tests

AgentState Model (Implemented)

class AgentState(TypedDict):
    ready_ids: list[str]                       # Subtask IDs ready to execute
    history: Annotated[list[dict], operator.add]  # Append-only execution history
    status: str                                # "running" | "done" | "failed"
    cost_so_far: float                         # Cumulative cost
# Note: TaskEntity held by reference on engine instance
# to avoid Pydantic strict-mode serialization issues

LangGraphTaskEngine (Implemented)

class LangGraphTaskEngine(TaskEngine):
    MAX_RETRIES = 2

    def _build_graph(self) -> CompiledStateGraph:
        graph = StateGraph(AgentState)
        graph.add_node("select_ready",   self._select_ready)    # Find ready subtasks
        graph.add_node("execute_batch",  self._execute_batch)   # Parallel via asyncio.gather
        graph.add_node("finalize",       self._finalize)

        graph.set_entry_point("select_ready")
        graph.add_edge("select_ready", "execute_batch")
        graph.add_conditional_edges(
            "execute_batch",
            self._route_after_execution,
            {"continue": "select_ready", "done": "finalize", "failed": "finalize"},
        )
        graph.add_edge("finalize", END)
        return graph.compile()

# Key design decisions:
# - Decomposition separated into IntentAnalyzer (called by CreateTaskUseCase)
# - Execution graph only handles subtask running (no decomposition nodes)
# - Independent subtasks execute in parallel via asyncio.gather
# - Failed subtasks retry up to MAX_RETRIES=2 then cascade failure to dependents

Completion Criteria

  • "Implement fibonacci in Python" → subtask decomposition → execution → result
  • Failure fallback: Ollama fails → retry with different model (MAX_RETRIES=2)
  • Parallel execution: 2 independent subtasks run simultaneously (asyncio.gather)
  • All tasks recorded via TaskRepository.save/update in use cases
  • Execution details recorded in AgentState.history (append-only)

Sprint 1.3b: Local Autonomous Execution Engine — LAEE (Day 8-9) — COMPLETE

Goal: Foundation for agent to directly operate user's local PC

Files Created

infrastructure/local_execution/__init__.py            # Re-exports
infrastructure/local_execution/executor.py            # LocalExecutor (risk→approve→execute→audit)
infrastructure/local_execution/audit_log.py           # JsonlAuditLogger (append-only JSONL)
infrastructure/local_execution/undo_manager.py        # Stack-based undo for reversible ops
infrastructure/local_execution/tools/__init__.py      # ToolRegistry (25 tools)
infrastructure/local_execution/tools/shell_tools.py   # shell_exec/background/stream/pipe
infrastructure/local_execution/tools/fs_tools.py      # fs_read/write/edit/delete/move/glob/tree
infrastructure/local_execution/tools/system_tools.py  # process_list/kill, resource, clipboard, notify
infrastructure/local_execution/tools/dev_tools.py     # git/docker/pkg_install/env_setup
tests/unit/infrastructure/test_local_execution.py     # 35 tests (all 8 completion criteria)

Note: Domain logic (RiskAssessor, ApprovalEngine) is already implemented in domain/services/. LAEE tools are thin wrappers around OSS/stdlib:

  • Shell → subprocess (stdlib)
  • File → pathlib (stdlib)
  • Browser → playwright (OSS)
  • Process → psutil (OSS)
  • Scheduling → apscheduler (OSS)
  • File watching → watchdog (OSS)

Completion Criteria

  • shell_exec("echo hello") → returns "hello"
  • fs_write + fs_read round-trip test
  • fs_delete(recursive=True) requires confirmation in confirm-destructive mode
  • full-auto mode executes all operations without confirmation
  • confirm-all mode requires confirmation for everything except SAFE
  • All operations logged to .morphic/audit_log.jsonl
  • undo_last() reverts a fs_write operation
  • Commands containing sudo auto-classified as CRITICAL

Integration Tests (10/10 pass with real Ollama)

tests/integration/test_live_smoke.py — requires Ollama running

TestOllamaLive:
  ✓ test_is_running           — Ollama health check
  ✓ test_list_models          — List installed models
  ✓ test_direct_inference     — Real qwen3 inference (think: false)

TestIntentAnalyzerLive:
  ✓ test_decompose_goal       — LLM decomposes goal → subtasks

TestLAEELive:
  ✓ test_shell_exec_real      — Real shell execution
  ✓ test_fs_workflow          — Write → Read → Edit → Undo → verify
  ✓ test_approval_modes       — 3 approval modes all work correctly
  ✓ test_sudo_detection       — sudo auto-classified as CRITICAL
  ✓ test_dev_git_status       — Real git status
  ✓ test_system_resource_info — CPU/memory/disk info

Verification: Cloud API Integration Tests (Step A) — COMPLETE

Date: 2026-02-25 All 4 providers verified. 11/11 tests pass. 181 unit tests + 11 cloud integration tests.

Test Results

Provider Model Tier Result Cost Notes
Anthropic claude-haiku-4-5-20251001 LOW PASS $0.000045 Cheapest API
Anthropic claude-sonnet-4-6 MEDIUM PASS $0.000135 Primary API model
Anthropic COMPLEX_REASONING route PASS Correctly routes to Sonnet
OpenAI o4-mini MEDIUM PASS $0.000104 O-series reasoning model
OpenAI o3 HIGH PASS $0.000190 O-series reasoning model
Gemini gemini-3-flash-preview LOW PASS $0.000010 Cheapest Gemini
Gemini gemini-3-pro-preview MEDIUM PASS $0.001264 Gemini Pro
Ollama qwen3-coder:30b FREE PASS $0.000000 Local, cost $0
Cost API call records cost > 0 PASS CostTracker verified
Cost Ollama records cost == 0 PASS CostTracker verified
Routing All providers route correctly PASS LOCAL_FIRST + task-type routing

Available Models (9 total across 4 providers)

FREE:   ollama/qwen3-coder:30b, ollama/qwen3:8b
LOW:    claude-haiku-4-5-20251001, gemini/gemini-3-flash-preview
MEDIUM: claude-sonnet-4-6, o4-mini, gemini/gemini-3-pro-preview
HIGH:   claude-opus-4-6, o3

Routing Results (LOCAL_FIRST=true, budget=$50)

simple_qa           -> ollama/qwen3-coder:30b  (FREE — local)
code_generation     -> ollama/qwen3-coder:30b  (FREE — local)
complex_reasoning   -> claude-sonnet-4-6       (MEDIUM — API)
file_operation      -> ollama/qwen3-coder:30b  (FREE — local)
long_context        -> claude-sonnet-4-6       (MEDIUM — API)
multimodal          -> claude-sonnet-4-6       (MEDIUM — API)

Issues Encountered & Fixed

Issue Root Cause Fix TD
Gemini 2.0 Flash 404 gemini-2.0-flash deprecated by Google Updated to gemini-3-*-preview TD-016
OpenAI temperature error O-series (o3, o4-mini) only supports temperature=1 Strip temperature param for O-series TD-016
OpenAI GPT-4o replaced User requested O-series models gpt-4o-minio4-mini, gpt-4oo3 TD-016
Codex API models codex-mini-latest is Responses API only (not Chat Completions) Use o3/o4-mini via standard Chat Completions API TD-016

Test File

tests/integration/test_cloud_llm.py — 11 tests
  _try_complete() helper: gracefully skips on auth/quota errors (pytest.skip)
  GEMINI_API_KEY env var: auto-set from settings for litellm compatibility

Next Steps

  • Step A: Cloud API Integration Tests — COMPLETE
  • Step B: E2E Pipeline Test — COMPLETE (below)
  • Step C: Sprint 1.4 Context Engineering

Verification: E2E Pipeline Test (Step B) — COMPLETE

Date: 2026-02-25 Full pipeline verified: Goal → LLM Decompose → DAG Execute → Result. 5/5 tests pass.

Test Results

Test LLM Subtasks Result Cost Key Verification
Create + Execute (is_prime) Ollama qwen3-coder:30b 5 (with deps) PASS $0.00 Full DAG with dependency chain
Parallel execution (fib+fact) Ollama qwen3-coder:30b 5 (2 independent) PASS $0.00 Independent subtasks run in parallel
Subtask results contain code Ollama qwen3-coder:30b 5 PASS $0.00 Actual Python code generated
Cost tracking ($0 local) Ollama qwen3-coder:30b 5 PASS $0.00 Sum of subtask costs == total_cost_usd
Cloud pipeline (Haiku) Claude Haiku 2 PASS $0.000512 Non-zero cost, model=claude-haiku

Pipeline Flow Verified

CreateTaskUseCase.execute(goal)
  └─ IntentAnalyzer.decompose(goal)     # LLM generates 2-5 subtasks with deps
     └─ TaskEntity created + persisted

ExecuteTaskUseCase.execute(task_id)
  └─ TaskRepository.get_by_id()         # Load from persistence
  └─ LangGraphTaskEngine.execute(task)
     └─ select_ready                    # Find subtasks with all deps completed
     └─ execute_batch                   # asyncio.gather for parallel execution
     └─ route_after_execution           # continue | done | failed
     └─ finalize                        # Set final status
  └─ TaskRepository.update()            # Persist final state

What Was Confirmed

  • LLM decomposition: IntentAnalyzer correctly breaks goals into atomic subtasks
  • Dependency resolution: Subtasks with deps wait for predecessors to complete
  • Parallel execution: Independent subtasks execute simultaneously via asyncio.gather
  • DAG state machine: select_ready → execute_batch → route cycles correctly
  • Cost tracking: Ollama=$0, Claude Haiku=$0.0005 accurately recorded
  • Persistence round-trip: save → get_by_id → update all work correctly
  • Status management: PENDING → RUNNING → SUCCESS/FALLBACK/FAILED transitions

Test File

tests/integration/test_e2e_pipeline.py — 5 tests (208s total, Ollama inference-bound)
  In-memory TaskRepository + CostRepository for persistence simulation
  TestE2EPipelineLocal: 4 tests with Ollama ($0)
  TestE2EPipelineCloud: 1 test with Claude Haiku (cost verification)

Sprint 1.4: Context Engineering (Day 10-11) — COMPLETE

Goal: Foundation of Manus 5 principles. KV-Cache optimization + tool masking + todo.md

Files Created

domain/services/tool_state_machine.py          # ToolStateMachine (mask/unmask, prefix ops)
domain/value_objects/tool_state.py             # ToolState enum, ToolDefinition model
infrastructure/context_engineering/__init__.py  # Re-exports
infrastructure/context_engineering/kv_cache_optimizer.py   # Stable prefix + deterministic serialization
infrastructure/context_engineering/observation_diversifier.py  # Template rotation
infrastructure/context_engineering/todo_manager.py         # FileTodoManager (auto-update)
infrastructure/context_engineering/file_context.py         # FileContext (hash-based cache)
tests/unit/infrastructure/test_context_engineering.py      # 40 tests

Completion Criteria

  • System prompt first 128 tokens are always identical (cache validation)
  • Tool definition count does not change during execution
  • todo.md auto-updated before/after task execution
  • 3 consecutive similar observations all serialized with different formats

Sprint 1.5: Semantic Memory (Day 12-13) — COMPLETE

Goal: L1-L4 memory hierarchy foundation. mem0 + pgvector + Neo4j integration

Files Created

domain/ports/knowledge_graph.py                # KnowledgeGraphPort ABC (L3 interface)
infrastructure/memory/__init__.py              # Re-exports
infrastructure/memory/memory_hierarchy.py      # MemoryHierarchy (L1-L4 unified manager)
infrastructure/memory/knowledge_graph.py       # Neo4jKnowledgeGraph (L3 Cypher adapter)
infrastructure/memory/context_zipper.py        # ContextZipper (query-adaptive compression)
tests/unit/infrastructure/test_memory.py       # 36 unit tests (in-memory fakes)
tests/integration/test_memory.py               # 8 integration tests (skip if services unavailable)

MemoryHierarchy (Implemented)

class MemoryHierarchy:
    """CPU-cache-inspired L1-L4 unified manager."""

    # L1: Active Context — collections.deque (bounded, O(1))
    # L2: Semantic Cache — MemoryRepository.search() (keyword/vector)
    # L3: Structured Facts — KnowledgeGraphPort.search_entities() (optional)
    # L4: Cold Storage — (via MemoryRepository, memory_type filter)

    async def add(self, content: str, role: str = "user") -> None:
        """L1: deque.append (always) + L2: memory_repo.add (persistent)"""

    async def retrieve(self, query: str, max_tokens: int = 500) -> str:
        """L1 → L2 → L3 scan, token-budget-aware, deduplication"""

ContextZipper (v1 → v2 in Sprint 3.2)

class ContextZipper:
    """v1: keyword-only scoring. v2 (Sprint 3.2, TD-030): async, semantic scoring,
    KG/memory augmentation, ingest() method. Optional ports for embedding/memory/KG."""

    async def compress(self, history: list[str], query: str, max_tokens: int = 500) -> str:
        """Score: recency(0.4) + semantic_similarity(0.6) → budget: [Facts]→[Memory]→[History]"""

    async def ingest(self, message: str, role: str = "user") -> None:
        """Store to L2 memory for future retrieval."""

Key Decisions

  • KnowledgeGraphPort is a domain port (like MemoryRepository): domain doesn't know about Neo4j
  • L1 uses collections.deque: bounded, O(1), no persistence
  • Knowledge graph optional: MemoryHierarchy works without Neo4j (L3 returns empty)
  • Token estimation: len(text) // 4 (Phase 1 approx), separator cost accounted in budget
  • Integration tests skip gracefully: @pytest.mark.skipif when PostgreSQL/Neo4j unavailable

Test Results

Unit tests:  36/36 pass (0.10s)
  TestMemoryHierarchy:      12 tests (add/retrieve, L1 priority, deque overflow, token budget, dedup)
  TestContextZipper:        10 tests (v1 backward compat, async)
  TestContextZipperV2:      16 tests (semantic scoring, KG/memory augmentation, ingest, multi-source)
  TestKnowledgeGraphPort:    5 tests (entity/relation CRUD, search, case-insensitive)
  TestCompletionCriteria:    5 tests (CC#1, CC#4 verification)
  TestEstimateTokens:        4 tests (helper function)

Integration tests: 8 tests (skip if services unavailable)
  TestPgvectorMemory:       3 tests (table exists, insert+query, pgvector extension)
  TestNeo4jKnowledgeGraph:  3 tests (add+search entity, add+query relation, case-sensitive search)
  TestMemoryHierarchyLive:  2 tests (end-to-end add+retrieve with real Neo4j)

Completion Criteria

  • add() → retrieve() returns relevant memories (5 CC#1 tests)
  • mem0 stores vectors in pgvector (3 integration tests, skip if no PG)
  • Neo4j stores entities/relations, searchable via Cypher (3 integration tests, skip if no Neo4j)
  • ContextZipper compresses 5000-token history → 500 tokens (2 CC#4 tests)

Sprint 1.6: API + UI (Day 14-15) — COMPLETE

Goal: FastAPI backend + Next.js minimal UI

Design Decisions

  • DI Pattern: AppContainer class (not FastAPI Depends chains). Stored on app.state.container. Simpler, testable, swappable
  • Background execution: POST /api/tasks decomposes synchronously, launches DAG execution via BackgroundTasks. No Celery
  • WebSocket MVP: Poll task_repo.get_by_id() every 1s, send JSON snapshots (delta-only) until is_complete
  • API schemas: Separate from domain entities. Plain strings for enums (no strict-mode issues in JSON)
  • Frontend: Next.js 15 App Router + Tailwind CSS. Bun package manager. Minimal functional components
  • No Docker/DB required: In-memory repos serve as Phase 1 production backend

Files Created

# Backend (Python)
domain/ports/task_repository.py            # +list_all() method
domain/ports/cost_repository.py            # +list_recent() method
infrastructure/persistence/in_memory.py    # InMemoryTaskRepository, InMemoryCostRepository, InMemoryMemoryRepository
interface/api/schemas.py                   # 10 Pydantic request/response models
interface/api/container.py                 # AppContainer DI wiring
interface/api/main.py                      # create_app() factory + lifespan + CORS
interface/api/websocket.py                 # /ws/tasks/{task_id} (poll + delta-only sends)
interface/api/routes/__init__.py
interface/api/routes/tasks.py              # POST, GET, GET/{id}, DELETE /api/tasks
interface/api/routes/models.py             # GET /api/models, GET /api/models/status
interface/api/routes/cost.py               # GET /api/cost, GET /api/cost/logs
interface/api/routes/memory.py             # GET /api/memory/search?q=
tests/unit/interface/__init__.py
tests/unit/interface/test_api.py           # 22 tests (TestClient + mock AppContainer)

# Frontend (TypeScript)
ui/                                        # Next.js 15 (bun, Tailwind CSS 4)
ui/lib/theme.ts                            # morphicAgentTheme design tokens
ui/lib/api.ts                              # Typed fetch wrappers + WebSocket client
ui/app/layout.tsx                          # Dark theme root layout (Geist font)
ui/app/globals.css                         # CSS variables matching design spec
ui/app/page.tsx                            # Dashboard (GoalInput + TaskList + sidebar)
ui/app/tasks/[id]/page.tsx                 # Task detail with live WebSocket updates
ui/components/GoalInput.tsx                # Textarea + Execute button (Enter to submit)
ui/components/TaskList.tsx                 # Task cards with status icons + FREE badge
ui/components/TaskDetail.tsx               # Subtask tree with status dots
ui/components/CostMeter.tsx                # Budget bar + daily/monthly/local stats
ui/components/ModelStatus.tsx              # Ollama status dot + model list

API Endpoints (15 total)

POST   /api/tasks              Create task → decompose → start background execution
GET    /api/tasks              List tasks (sorted by created_at desc)
GET    /api/tasks/{id}         Task detail (subtasks, cost, success_rate)
DELETE /api/tasks/{id}         Delete task

GET    /api/models             List available model names
GET    /api/models/status      Ollama health + model list + default model

GET    /api/cost               Cost summary (daily/monthly/local rate/budget)
GET    /api/cost/logs          Recent cost log entries

GET    /api/memory/search?q=   Semantic memory search via MemoryHierarchy

GET    /api/health             Health check

WS     /ws/tasks/{id}          Real-time task snapshots (1s poll, delta-only)

AppContainer DI Wiring

Settings → OllamaManager → CostTracker → LiteLLMGateway
         → IntentAnalyzer → LangGraphTaskEngine
         → CreateTaskUseCase, ExecuteTaskUseCase
         → InMemory*Repository (task, cost, memory)
         → MemoryHierarchy

Test Results

22 API tests (TestClient + mock AppContainer):
  TestTaskEndpoints:     9 tests (CRUD, validation, subtask response)
  TestModelEndpoints:    3 tests (list, status, ollama down)
  TestCostEndpoints:     3 tests (summary empty, with records, logs)
  TestMemoryEndpoints:   2 tests (empty, with data)
  TestWebSocket:         3 tests (not found, snapshot, stop on complete)
  TestApp:               2 tests (health, CORS headers)

Total unit test suite: 257 → 279 tests, all passing (1.71s)
Next.js build: 0 TypeScript errors, static + dynamic routes

Completion Criteria

  • POST /api/tasks sends goal → task decomposition + background execution starts
  • WebSocket receives real-time progress (delta-only snapshots)
  • Next.js UI displays dashboard with task list + detail page
  • Cost meter displayed ($0.00 / Local 100% / budget bar)
  • 22 backend tests pass (TestClient + mock LLM)
  • next build succeeds with 0 TypeScript errors

Sprint 1.7: Integration & E2E Test (Day 16) ✅ COMPLETE

Goal: Full component integration test. Validate $0 path.

Completed: 2026-02-25 | New tests: 19 (7 failure recovery + 12 API E2E) | Total: 298 unit tests (1.72s)

Test Scenarios — Coverage Analysis

# Scenario Status Location
E2E Test 1 $0 Full Local Path ✅ Covered (Sprint 1.4) tests/integration/test_e2e_pipeline.py::TestE2EPipelineLocal (4 tests)
E2E Test 2 Failure Recovery NEW tests/unit/infrastructure/test_failure_recovery.py (7 tests)
E2E Test 3 Parallel Execution ✅ Covered (Sprint 1.4) test_e2e_pipeline.py::test_parallel_subtask_execution
E2E Test 4 Memory Persistence ✅ Covered (Sprint 1.5) tests/unit/infrastructure/test_memory.py::TestMemoryHierarchy (12 tests)
E2E Test 5 LAEE Local Execution ✅ Covered (Sprint 1.3) tests/unit/infrastructure/test_local_execution.py (10+ tests)
E2E Test 6 LAEE Approval Mode ✅ Covered (Sprint 1.3) tests/unit/domain/test_approval_engine.py
E2E Test 7 LAEE Undo ✅ Covered (Sprint 1.3) tests/unit/infrastructure/test_local_execution.py::TestUndoManager

New Test Details

Failure Recovery (7 tests)tests/unit/infrastructure/test_failure_recovery.py:

TestFailureRecoveryRetry:
  ✓ test_retry_then_succeed        — LLM fails once, retry succeeds → SUCCESS
  ✓ test_retry_exhausted_then_fail — All retries fail → FAILED

TestFailureRecoveryPartialSuccess:
  ✓ test_partial_success_fallback  — 1 success + 1 fail → FALLBACK

TestFailureRecoveryDependencyBlocking:
  ✓ test_dependency_cascade_failure       — A fails → B blocked → both FAILED
  ✓ test_dependency_chain_success_then_fail — A succeeds → B fails → FALLBACK

TestFailureRecoveryPersistence:
  ✓ test_failed_state_persisted    — Failed state correctly persisted in repo
  ✓ test_retry_success_persisted   — Recovered state correctly persisted

API E2E (12 tests)tests/unit/interface/test_api_e2e.py:

TestAPIEndToEnd:
  ✓ test_create_task_returns_201_with_subtasks — POST → 201 + subtask list
  ✓ test_post_then_get_shows_completed         — POST → BackgroundTasks → GET → success
  ✓ test_list_tasks_after_creation             — Multiple POST → GET /api/tasks → all listed
  ✓ test_delete_after_execution                — POST → DELETE → 404

TestAPIEndToEndFailure:
  ✓ test_create_task_empty_goal_422, test_create_task_missing_goal_422
  ✓ test_get_nonexistent_task_404, test_delete_nonexistent_task_404

TestAPIEndToEndCostTracking:
  ✓ test_cost_summary_reflects_execution — Local $0 + 100% local rate
  ✓ test_cost_logs_contain_records       — Cloud cost log entry

TestAPIEndToEndWebSocket:
  ✓ test_ws_reflects_completed_task — WS sees completed state after POST
  ✓ test_ws_nonexistent_task_error  — WS returns error for missing task

Completion Criteria

  • E2E Tests 1-7 all covered (5 from prior sprints + 2 new test files)
  • uv run pytest tests/unit/ -v → 298 tests pass (1.72s, no Docker required)
  • API round-trip: POST /api/tasks → BackgroundTasks execute → GET shows completion

Phase 1 Deliverable Summary ✅ COMPLETE

Status: ALL 7 SPRINTS COMPLETE (1.1 → 1.7)
Tests:  298 unit tests (1.72s) + 5 integration tests (Ollama required)
Files:  ~70 Python + ~11 TypeScript

Python packages (installed):
  langgraph, litellm, fastapi, uvicorn, pydantic-settings,
  httpx, pytest, pytest-asyncio
  (DB/queue packages deferred to Phase 2 — in-memory repos for Phase 1)

Next.js packages (installed via bun):
  next 15, react 19, tailwindcss 4

Verification:
  ✓ User inputs goal → DAG generated → Ollama executes → results displayed
  ✓ Cost: $0 (full local execution path verified)
  ✓ Failure recovery: retry + fallback tested
  ✓ API round-trip: POST → BackgroundTasks → GET → completed task
  ✓ WebSocket: real-time task progress
  ✓ Next.js UI: dashboard + task detail + cost meter

Phase 2: Parallel & Planning + CLI v1 (Week 3-4) ✅ COMPLETE

Goal: Full parallel execution + Interactive Planning + CLI foundation

Result: All 6 sprints (2-A through 2-F) + CLI v1 (2.9-2.11) complete. 428 unit tests, all passing (2.48s).

Week 3: Parallel Execution & Interactive Planning

# Item File Status
2.1 ParallelExecutionEngine full impl infrastructure/task_graph/engine.py (asyncio.gather) ✅ Phase 1
2.2 Celery worker integration infrastructure/queue/celery_app.py, tasks.py ✅ Sprint 2-B
2.3 Interactive Planning System application/use_cases/interactive_plan.py ✅ Sprint 2-C
2.4 Cost estimation engine application/use_cases/cost_estimator.py ✅ Sprint 2-C

Interactive Planning Flow:

1. User inputs goal
2. LLM decomposes into subtasks + proposes models
3. Cost estimate calculated per-step (ollama/* = $0)
4. Plan + estimate presented in UI (PlanningView) / CLI (morphic plan)
5. User [approve / reject]
6. Approve → creates TaskEntity → execution starts

Week 4: Background Planner, Graph Viz & CLI

# Item File Status
2.5 Background Planner (Windsurf-style) application/use_cases/background_planner.py ✅ Sprint 2-D
2.6 Tool State Machine enhancement domain/services/risk_assessor.py (existing) ✅ Phase 1
2.7 React Flow task graph UI ui/components/TaskGraph.tsx ✅ Sprint 2-F
2.8 Planning View UI ui/components/PlanningView.tsx ✅ Sprint 2-F
2.9 CLI foundation (typer + rich) interface/cli/main.py ✅ Sprint 2.9
2.10 CLI task commands interface/cli/commands/task.py ✅ Sprint 2.10
2.11 CLI model/cost commands interface/cli/commands/model.py, cost.py ✅ Sprint 2.11
2.12 LAEE Browser Tools (Playwright) infrastructure/local_execution/tools/browser_tools.py ✅ Sprint 2-E
2.13 LAEE GUI Tools (macOS) infrastructure/local_execution/tools/gui_tools.py ✅ Sprint 2-E
2.14 LAEE Cron Tools (APScheduler) infrastructure/local_execution/tools/cron_tools.py ✅ Sprint 2-E

Phase 2 Completion Criteria:

  • 3 independent tasks execute in parallel, 3x+ faster than sequential ✅ (asyncio.gather in Phase 1 engine)
  • Interactive Planning: plan presented → user approves → execution starts ✅ Sprint 2-C
  • React Flow visualizes DAG in real-time ✅ Sprint 2-F
  • Background Planner continuously improves plan during execution ✅ Sprint 2-D
  • morphic task create "..." creates and executes task from CLI ✅ Sprint 2.9-2.10
  • morphic cost summary displays cost breakdown in terminal ✅ Sprint 2.11
  • PostgreSQL repos replace InMemory (opt-in via USE_POSTGRES=true) ✅ Sprint 2-A
  • Celery worker for async task execution (opt-in via CELERY_ENABLED=true) ✅ Sprint 2-B
  • LAEE browser/gui/cron tools (14 new tools → 36 total in TOOL_REGISTRY) ✅ Sprint 2-E

Sprint 2-A: PG Repos + Alembic (2026-02-26)

Delivered: PostgreSQL repository implementations + Alembic initial migration.

Deliverable Description
pg_task_repository.py PgTaskRepository mapping TaskEntity ↔ TaskModel (subtasks as JSONB)
pg_cost_repository.py PgCostRepository with SQL aggregation for daily/monthly/local stats
pg_memory_repository.py PgMemoryRepository with ILIKE keyword search
pg_plan_repository.py PgPlanRepository mapping ExecutionPlan ↔ PlanModel
001_initial_schema.py Alembic migration: tasks, task_executions, memories, cost_logs, plans
container.py updated _create_repos() switches PG/InMemory via Settings.use_postgres

Tests: 19 new tests (mocked async sessions). Tech Decisions: TD-024 (PG/InMemory switching).

Sprint 2-B: Celery + Redis Worker (2026-02-26)

Delivered: Celery-based async task execution.

Deliverable Description
celery_app.py Celery app factory (Redis broker + backend)
tasks.py execute_task_worker task creates own AppContainer
routes/tasks.py updated Celery dispatch when celery_enabled=True

Tests: 7 new tests. Tech Decisions: TD-025 (Celery gated by settings flag).

Sprint 2-C: Cost Estimation + Interactive Planning (2026-02-26)

Delivered: Full interactive planning system with cost estimation.

Deliverable Description
plan.py (entity) PlanStep, ExecutionPlan domain entities
PlanStatus enum proposed/approved/rejected/executing/completed
cost_estimator.py MODEL_COST_TABLE, per-step cost estimation
interactive_plan.py create_plan/approve_plan/reject_plan use cases
routes/plans.py POST/GET/approve/reject API endpoints
commands/plan.py CLI: morphic plan create/list/show/approve/reject

Tests: 38 new tests. Tech Decisions: TD-026 (PlanStatus enum), TD-027 (cost estimation model).

Sprint 2-D: Background Planner (2026-02-26)

Delivered: Advisory background planner for running tasks.

Deliverable Description
background_planner.py Start/stop monitoring, failure recommendations
websocket.py updated Recommendations included in WS snapshots

Tests: 10 new tests.

Sprint 2-E: LAEE Browser/GUI/Cron Tools (2026-02-26)

Delivered: 14 new LAEE tools (36 total in TOOL_REGISTRY).

Module Tools
browser_tools.py navigate, click, type, screenshot, extract, pdf (Playwright)
gui_tools.py applescript, open_app, screenshot_ocr, accessibility (macOS)
cron_tools.py schedule, once, list, cancel (APScheduler)

Tests: 34 new tests (all mocked — no real Playwright/AppleScript).

Sprint 2-F: React Flow + Planning UI (2026-02-26)

Delivered: Visual DAG + plan review/approve UI.

Deliverable Description
TaskGraph.tsx React Flow DAG with SubTaskNode, status colors, FREE badge
PlanningView.tsx Plan steps table + cost display + approve/reject buttons
plans/[id]/page.tsx Plan detail page
page.tsx updated Execute/Plan First mode toggle
api.ts updated Plan types + API functions

Tests: TypeScript build verified (0 errors). Package: @xyflow/react.

Sprint 2.9-2.11 Results: CLI v1 Complete (2026-02-25)

Delivered: Full morphic CLI with 4 subcommand groups, 15 commands total.

Command Description Status
morphic --version Show version (morphic-agent 0.4.0a0)
morphic --help Show all subcommands
morphic task create "goal" Create + execute with spinner (real Ollama)
morphic task create "goal" --no-wait Create only, return ID
morphic task list Rich table of all tasks
morphic task show <id> Tree view with subtask results
morphic task cancel <id> Set status to FAILED
morphic model list Table of installed Ollama models + LOCAL tag
morphic model status Ollama Running/Stopped + default model
morphic model pull <name> Pull with spinner
morphic cost summary Daily/monthly/local rate/budget table
morphic cost budget <amount> Set monthly budget (in-memory)
morphic plan create "goal" Create plan with cost estimate
morphic plan list Rich table of all plans
morphic plan show <id> Plan detail + steps table

Tests: 24 CLI tests (3 foundation + 9 task + 5 model + 3 cost + 4 plan).

Tech Decisions: TD-021 (AppContainer reuse), TD-022 (sync test strategy), TD-023 (cross-process data loss), TD-024–TD-027 (PG repos, Celery, plans).

Known Limitation: morphic task list returns empty after morphic task create --no-wait because each CLI invocation is a separate process with in-memory repos. Resolution: set USE_POSTGRES=true with Docker Compose running. See TD-023.


Pre-Phase 3 Verification (2026-02-26) ✅ COMPLETE

Full codebase health check before starting Phase 3.

Results

Check Result Notes
Unit Tests 428 passed (2.82s) 3 warnings (PG mock coroutines — cosmetic)
Integration Tests 10 passed (15.69s) Real Ollama (qwen3-coder:30b, qwen3:8b)
Ruff Lint 0 errors (79 fixed) See TD-028 for details
Ruff Format 139 files clean (48 reformatted) Consistent style enforced
FastAPI Server /api/health OK, /api/models/status OK 9 models visible (2 local + 7 cloud)
CLI Import OK interface.cli.main:app loads without error
Ollama Running qwen3-coder:30b (default), qwen3:8b
Next.js Build 0 TypeScript errors 4 routes (/ + /_not-found + /plans/[id] + /tasks/[id])
Docker Compose Not running PG/Redis/Neo4j not needed for default InMemory mode

Pre-Phase 3 Cleanup

  • 79 ruff lint errors → 0: unused imports, unsorted imports, line length, exception chaining, contextlib.suppress, unused variables, collapsible if/with (TD-028)
  • 48 files reformatted: consistent style with ruff format
  • dev dependencies restored: uv sync --extra dev (ruff, pytest, mypy, pytest-cov)

Phase 3 Readiness

Prerequisite Status
Phase 2 complete (all sprints)
All tests passing ✅ 428 unit + 10 integration
Lint clean ✅ ruff check + format
Ollama operational ✅ 2 models ready
Memory hierarchy (L1-L4) foundation ✅ Sprint 1.5
ContextZipper v1 → v2 ✅ Sprint 1.5 (v1, keyword) → Sprint 3.2 (v2, semantic, TD-030)
KnowledgeGraphPort defined ✅ domain/ports/

Phase 3: Context Bridge & Semantic Memory (Week 5-6)

Goal: Elevate memory and context to research-grade + cross-platform support

Sprint 3.1: SemanticFingerprint (LSH) (2026-02-26) ✅ COMPLETE

LSH-based semantic search with embeddings, replacing keyword-only search.

What Was Built

Component File Description
EmbeddingPort domain/ports/embedding.py ABC: embed(texts), dimensions()
SemanticFingerprint domain/services/semantic_fingerprint.py LSH hash + cosine similarity (pure numpy, no I/O)
SemanticBucketStore infrastructure/memory/semantic_fingerprint.py LSH bucketing + multi-probe retrieval
OllamaEmbeddingAdapter infrastructure/memory/embedding_adapters.py POST /api/embed (LOCAL_FIRST, $0)
Migration 002 migrations/versions/002_add_embedding_column.py Vector(384) + HNSW index

Modified Files

File Change
shared/config.py 5 new embedding settings (embedding_backend, embedding_model, embedding_dimensions, embedding_lsh_seed, embedding_lsh_n_planes)
infrastructure/persistence/models.py Vector(1536)Vector(384)
infrastructure/persistence/in_memory.py Optional embedding_port param, vector search with SemanticBucketStore
infrastructure/persistence/pg_memory_repository.py Optional embedding_port, pgvector cosine_distance search
interface/api/container.py DI wiring: _create_embedding_port() + pass to repos
domain/ports/__init__.py Export EmbeddingPort
pyproject.toml numpy>=1.26 explicit dep

Test Results

Suite Tests Status
test_semantic_fingerprint.py 11 ✅ LSH hash, cosine sim, determinism, granularity
test_semantic_search.py 20 ✅ BucketStore, OllamaAdapter, InMemory vector search
All existing tests 428 ✅ Full backward compatibility
Total 459 All passing (2.52s)

Key Design Decisions (TD-029)

  • Ollama embedding: LOCAL_FIRST, $0, all-minilm 384-dim
  • Seeded RNG: seed=42 for deterministic LSH across restarts
  • Domain purity: MemoryEntry stays clean; vectors only in ORM layer
  • Backward compat: embedding_port=None → keyword fallback

Sprint 3.2: ContextZipper v2 — Semantic-Aware Compression (2026-02-26) ✅ COMPLETE

Rewrite ContextZipper from sync keyword-only to async semantic-aware compressor with memory/KG augmentation.

What Was Built

Component File Description
ContextZipper v2 infrastructure/memory/context_zipper.py async compress(), semantic scoring, [Facts]/[Memory] augmentation, ingest()

Modified Files

File Change
infrastructure/memory/context_zipper.py Full rewrite: 84-line sync → 254-line async. Optional ports, budget allocation, per-word entity search
tests/unit/infrastructure/test_memory.py 10 existing tests → async. 16 new v2 tests (constructor, semantic, KG, memory, ingest, multi-source)
interface/api/container.py Wire ContextZipper with embedding_port + memory_repo

API

class ContextZipper:
    def __init__(self, embedding_port?, memory_repo?, knowledge_graph?,
                 facts_budget_pct=0.20, memory_budget_pct=0.30): ...
    async def compress(self, history, query, max_tokens=500) -> str: ...
    async def ingest(self, message, role="user") -> None: ...

Test Results

Suite Tests Status
TestContextZipper (v1 backward compat) 10 ✅ async, same behavior
TestContextZipperV2 (new features) 16 ✅ semantic, KG, memory, ingest
All existing tests 459 ✅ Full backward compatibility
Total 475 All passing (2.98s)

Key Design Decisions (TD-030)

  • Semantic fallback: cosine similarity (with EmbeddingPort) → keyword overlap (without)
  • Budget allocation: [Facts] 20% → [Memory] 30% → [History] 50% (configurable)
  • Per-word entity search: multi-word queries split by word, deduplicated by entity ID
  • Deduplication: memory entries matching history messages are skipped
  • No new deps: reuses existing EmbeddingPort, MemoryRepository, KnowledgeGraphPort

Sprint 3.3: ForgettingCurve — Ebbinghaus Retention Scoring (2026-02-26) ✅ COMPLETE

Automatic L2 memory expiration via retention scoring. Expired entries promoted to L3 KG then deleted.

What Was Built

Component File Description
ForgettingCurve domain/services/forgetting_curve.py Pure math: retention_score, is_expired, hours_since. R=e^(-t/S)
ForgettingCurveManager infrastructure/memory/forgetting_curve.py Async manager: scan L2 → expire → promote to KG → delete
CompactResult infrastructure/memory/forgetting_curve.py Frozen dataclass: scanned/expired/promoted/deleted counts

Modified Files

File Change
domain/ports/memory_repository.py Added list_by_type(memory_type, limit) abstract method
infrastructure/persistence/in_memory.py Implemented list_by_type (filter _store.values())
infrastructure/persistence/pg_memory_repository.py Implemented list_by_type (SQL WHERE + ORDER BY last_accessed ASC)
infrastructure/memory/memory_hierarchy.py Added compact(threshold) delegating to ForgettingCurveManager
interface/api/container.py Wired ForgettingCurveManager with memory_repo + settings threshold

Test Results

Suite Tests Status
test_forgetting_curve.py (domain) 14 ✅ retention_score, is_expired, hours_since, boundary, stability
test_forgetting_curve.py (infrastructure) 17 ✅ compact, promote, score_entry, list_by_type, CompactResult
All existing tests 475 ✅ Full backward compatibility
Total 506 All passing (2.8s)

Key Design Decisions (TD-031)

  • Pure domain service: retention_score/is_expired/hours_since are static, no I/O
  • Strict less-than: score < threshold — entries exactly at boundary are NOT expired (conservative)
  • KG optional: Without KG, expired entries are simply deleted (graceful degradation per TD-017)
  • No LLM: Promotion stores content directly as memory_fact entity ($0 cost)
  • Reuses existing config: Settings.memory_retention_threshold=0.3 already existed

Sprint 3.4: DeltaEncoder — Git-Style Delta State Tracking (2026-02-26) ✅ COMPLETE

Git-style delta encoding for state tracking. Records changes as diffs, reconstructs any point-in-time state.

What Was Built

Component File Description
Delta domain/entities/delta.py Pydantic entity (strict): topic, seq, message, changes, state_hash
DeltaEncoder domain/services/delta_encoder.py Pure static: hash_changes, reconstruct, create_delta, compute_diff
DeltaEncoderManager infrastructure/memory/delta_encoder.py Async manager: record, get_state, get_history, list_topics
DeltaRecordResult infrastructure/memory/delta_encoder.py Frozen dataclass: delta_id, topic, seq, state_hash

Modified Files

File Change
domain/entities/__init__.py Export Delta
infrastructure/memory/__init__.py Export DeltaEncoderManager
infrastructure/memory/memory_hierarchy.py Added record_delta(), get_state(), get_state_history()
interface/api/container.py Wired DeltaEncoderManager with memory_repo

Test Results

Suite Tests Status
test_delta_encoder.py (domain) 34 ✅ entity validation, hash, reconstruct, create_delta, compute_diff
test_delta_encoder.py (infrastructure) 27 ✅ record, get_state, history, topics, roundtrip, hierarchy integration
All existing tests 506 ✅ Full backward compatibility
Total 567 All passing (2.8s)

Key Design Decisions (TD-032)

  • Zero new ports: Deltas stored as MemoryEntry (L2_SEMANTIC) with delta_* metadata keys
  • Topic-based grouping: metadata["delta_topic"] for namespace isolation
  • SHA-256 deterministic hash: json.dumps(sort_keys=True) — Manus principle 1
  • Tombstone deletion: compute_diff uses None value for deleted keys
  • Auto-seq + auto-base: First delta per topic is automatically seq=0 and is_base_state=True
  • No new deps: hashlib + json are stdlib

Sprint 3.5: HierarchicalSummarizer — Multi-Level Tree Compression (2026-02-26) ✅ COMPLETE

4-level tree compression for memory entries. Query-adaptive retrieval selects the deepest level that fits within a token budget.

What Was Built

Component File Description
HierarchicalSummarizer domain/services/hierarchical_summarizer.py Pure static: estimate_tokens, split_sentences, extract_summary, build_extractive_hierarchy, select_level, estimate_depth
HierarchicalSummaryManager infrastructure/memory/hierarchical_summarizer.py Async manager: summarize, get_summary, retrieve_at_depth. Optional LLM for abstractive, extractive fallback
SummarizeResult infrastructure/memory/hierarchical_summarizer.py Frozen dataclass: entry_id, levels_built, original_tokens, compressed_tokens, used_llm

Modified Files

File Change
infrastructure/memory/__init__.py Export HierarchicalSummaryManager
infrastructure/memory/memory_hierarchy.py Added summarize_entry(), retrieve_at_depth()
interface/api/container.py Wired HierarchicalSummaryManager with memory_repo + optional LLM

Test Results

Suite Tests Status
test_hierarchical_summarizer.py (domain) 27 ✅ tokens, sentences, extract, hierarchy, select_level, estimate_depth
test_hierarchical_summarizer.py (infrastructure) 24 ✅ extractive, LLM, get_summary, retrieve_at_depth, skip-existing, integration
All existing tests 567 ✅ Full backward compatibility
Total 618 All passing (2.8s)

Key Design Decisions (TD-033)

  • Zero new ports: Summaries stored in MemoryEntry.metadata (hierarchy_summaries, hierarchy_token_counts JSON keys)
  • 4 fixed levels: L0=100% (original), L1=~40%, L2=~15%, L3=~5% of sentences
  • Optional LLM: Abstractive summaries when available, sentence-boundary extractive fallback when not
  • Skip re-summarization: If hierarchy_summaries already exists in metadata, returns cached result
  • Depth-adaptive retrieval: select_level() finds deepest level fitting within token budget
  • No new deps: re + math are stdlib

Week 5: Full Semantic Memory

# Item File
3.1 SemanticFingerprint (LSH) infrastructure/memory/semantic_fingerprint.py
3.2 ContextZipper v2 (semantic) infrastructure/memory/context_zipper.py
3.3 ForgettingCurve infrastructure/memory/forgetting_curve.py
3.4 DeltaEncoder infrastructure/memory/delta_encoder.py
3.5 HierarchicalSummarizer infrastructure/memory/hierarchical_summarizer.py

Week 6: Context Bridge & MCP

# Item File
3.6 Cross-Platform Context Bridge infrastructure/memory/context_bridge.py
3.7 MCP Server implementation infrastructure/mcp/server.py
3.8 MCP Client implementation infrastructure/mcp/client.py
3.9 Chrome Extension integrations/browser_extension/
3.10 L1→L4 integration test tests/integration/test_memory_hierarchy.py

Phase 3 Completion Criteria:

  • 10,000 tokens → 500 tokens compression (information retention > 90%) — ContextZipper v2 + HierarchicalSummarizer
  • LSH retrieves semantically similar memories in near-O(1) time — SemanticFingerprint
  • MCP Server enables other tools to access Morphic-Agent memory
  • Forgetting curve auto-promotes low-importance memories to L3, removes from L2
  • Delta encoding tracks state changes as diffs, reconstructs any point-in-time state
  • Hierarchical summarization provides 4-level tree compression with depth-adaptive retrieval

Phase 4: Agent CLI Orchestration (Week 7-8)

Goal: Meta-orchestrator managing 4 Agent CLIs

Week 7: Common Interface + OpenHands & Claude Code

# Item File
4.1 AgentEngine Protocol infrastructure/agent_orchestration/agent_engine_protocol.py
4.2 OpenHands Driver infrastructure/agent_orchestration/openhands_driver.py
4.3 Claude Code SDK Driver infrastructure/agent_orchestration/claude_code_driver.py
4.4 AgentCLIRouter foundation application/use_cases/agent_routing.py

Week 8: Gemini & Codex + Router completion

# Item File
4.5 Gemini CLI + ADK Driver infrastructure/agent_orchestration/gemini_adk_driver.py
4.6 Codex CLI Driver infrastructure/agent_orchestration/codex_cli_driver.py
4.7 AgentCLIRouter routing complete (extend 4.4)
4.8 Knowledge file management infrastructure/agent_orchestration/knowledge_files.py

Phase 4 Completion Criteria:

  • Same task executed on OpenHands / Claude Code / Gemini / Codex / ADK, results compared
  • AgentCLIRouter auto-selects optimal engine based on task characteristics
  • Availability check + fallback for each engine
  • ADK Driver (Google ADK Python SDK) with try-import guard (Sprint 4.5)
  • Knowledge file management — engine-specific context injection (Sprint 4.6)

Phase 5: Marketplace & Tools (Week 9-10)

Goal: Autonomous tool discovery, installation, and sharing

# Item File
5.1 Auto Tool Discoverer infrastructure/marketplace/auto_discoverer.py
5.2 MCP Registry search infrastructure/marketplace/mcp_search.py
5.3 Tool Installer infrastructure/marketplace/tool_installer.py
5.4 Ollama Model Manager infrastructure/marketplace/ollama_installer.py
5.5 Tool Safety Scorer infrastructure/marketplace/safety_scorer.py
5.6 Marketplace UI ui/app/marketplace/page.tsx
5.7a Model Management Test Coverage tests/unit/interface/test_api.py, tests/unit/interface/test_cli.py
5.7b Auto-Discovery Trigger on Task Failure application/use_cases/execute_task.py, tests/unit/application/test_execute_task.py

Phase 5 Completion Criteria: ✅ ALL MET

  • Auto-search and suggest tools on task failure — Sprint 5.7b (_safe_suggest_tools in ExecuteTaskUseCase)
  • 1-click install from MCP Registry — Sprint 5.3 (InstallToolUseCase + API + CLI)
  • Ollama model UI management (pull/delete/switch) — Sprint 5.5 + 5.6 + 5.7a (ManageOllamaUseCase + 15 interface tests)
  • Tool safety score displayed — Sprint 5.1 + 5.6 (ToolSafetyScorer + SafetyBadge)

Phase 6: Self-Evolution (Week 11-12) ✅ COMPLETE

Goal: Autonomous improvement from execution data

Result: 5/5 sprints complete. 1162 unit tests (0 failures), lint clean. 3-tier evolution engine fully operational.

# Item File Status
6.1 Execution Recorder + Domain Foundation domain/entities/execution_record.py, domain/value_objects/evolution.py, domain/ports/execution_record_repository.py, application/use_cases/analyze_execution.py
6.2 Tactical Recovery (Level 1) domain/entities/strategy.py, domain/services/tactical_recovery.py
6.3 Strategy Updater (Level 2) application/use_cases/update_strategy.py, infrastructure/evolution/strategy_store.py
6.4 Systemic Evolver (Level 3) application/use_cases/systemic_evolution.py
6.5 Evolution Interface (API + CLI + UI) interface/api/routes/evolution.py, interface/cli/commands/evolution.py, ui/app/evolution/page.tsx

Phase 6 Completion Criteria:

  • Failure pattern analysis → FailureAnalyzer + AnalyzeExecutionUseCase.get_failure_patterns()
  • Model/engine preference learning → UpdateStrategyUseCase.run_full_update()
  • Agent CLI engine selection auto-optimized → EnginePreference tracking per task type
  • Evolution reports viewable in UI → /api/evolution/evolve + UI dashboard
  • Bonus: Fixed 19 pre-existing test_mcp_server.py failures

Phase 7: Unified Cognitive Layer + Meta-Orchestration v2 (Week 13-16)

Goal: Shared memory + shared task state across all agent engines. Any agent can see, continue, and build on any other agent's work. The "memory hub" that makes the whole greater than the sum of parts.

Key Insight: A2A (task delegation) is necessary but insufficient. The real value is shared cognition — agents that share understanding, not just tasks.

Sprint 7.1: UCL Domain Foundation — COMPLETE (2026-03-10)

Domain entities, value objects, and ports for the Unified Cognitive Layer. 68 tests passed, 0 failures. Total: 1230 unit tests.

# Item File Status
1 CognitiveMemoryType value object domain/value_objects/cognitive.py
2 Decision entity domain/entities/cognitive.py
3 AgentAction entity domain/entities/cognitive.py
4 SharedTaskState entity domain/entities/cognitive.py
5 AgentAffinityScore entity domain/entities/cognitive.py
6 SharedTaskStateRepository port domain/ports/shared_task_state_repository.py
7 InsightExtractorPort port + ExtractedInsight domain/ports/insight_extractor.py
8 AgentAffinityScorer domain service domain/services/agent_affinity.py
9 Tests (68) tests/unit/domain/test_cognitive.py

Sprint 7.2: Shared Task State + Context Adapters — COMPLETE (2026-03-10)

Infrastructure: persist shared task state + bidirectional context translation per engine. 120 tests passed, 0 failures. Total: 1350 unit tests.

# Item File Status
1 ContextAdapterPort port + AdapterInsight domain/ports/context_adapter.py
2 InMemorySharedTaskStateRepository infrastructure/persistence/shared_task_state_repo.py
3 _base.py shared helpers infrastructure/cognitive/adapters/_base.py
4 ClaudeCodeContextAdapter infrastructure/cognitive/adapters/claude_code.py
5 GeminiContextAdapter infrastructure/cognitive/adapters/gemini.py
6 CodexContextAdapter infrastructure/cognitive/adapters/codex.py
7 OllamaContextAdapter infrastructure/cognitive/adapters/ollama.py
8 OpenHandsContextAdapter infrastructure/cognitive/adapters/openhands.py
9 ADKContextAdapter infrastructure/cognitive/adapters/adk.py
10 Repo tests (16) tests/unit/infrastructure/test_shared_task_state_repo.py
11 Adapter tests (104) tests/unit/infrastructure/test_context_adapters.py

Sprint 7.3: Insight Extraction Pipeline

Post-execution pipeline: agent output → structured insights → UCL memory + task state update.

# Item File Notes
1 InsightExtractor implementation infrastructure/cognitive/insight_extractor.py Regex + keyword extraction (LLM-enhanced later)
2 ConflictResolver domain service domain/services/conflict_resolver.py Confidence-weighted, recency-biased, detects contradictions
3 ExtractInsightsUseCase application/use_cases/extract_insights.py Orchestrates: extract → conflict check → store → update task state
4 MemoryClassifier domain service domain/services/memory_classifier.py Auto-classify into EPISODIC/SEMANTIC/PROCEDURAL/WORKING
5 Integration with ExecuteTaskUseCase application/use_cases/execute_task.py Post-execution hook: auto-extract + auto-store
6 Tests tests/unit/application/test_extract_insights.py Extraction accuracy, conflict detection, classification

Sprint 7.4: Affinity-Aware Routing + Task Handoff

Extend AgentEngineRouter with affinity scoring. Enable cross-agent task handoff.

# Item File Notes
1 Extend AgentEngineRouter domain/services/agent_engine_router.py Add affinity_score to routing decision (alongside cost/capability)
2 HandoffTaskUseCase application/use_cases/handoff_task.py Capture state from Agent A → prepare context for Agent B → delegate
3 AffinityStore infrastructure/cognitive/affinity_store.py JSONL persistence (topic → engine → scores), updated after each execution
4 Update RouteToEngineUseCase application/use_cases/route_to_engine.py Inject shared context via ContextAdapter before engine execution
5 Tests tests/unit/application/test_handoff.py Handoff preserves decisions, artifacts, blockers

Sprint 7.5: UCL API + CLI + UI ✅ (2026-03-11)

Expose UCL capabilities through all interfaces.

# Item File Status
1 UCL API schemas interface/api/schemas.py ✅ 12 schemas (DecisionResponse, SharedTaskStateResponse, AffinityScoreResponse, HandoffRequestSchema, InsightResponse, etc.)
2 UCL API endpoints interface/api/routes/cognitive.py ✅ 6 endpoints (state CRUD, affinity, handoff, insights/extract)
3 UCL CLI commands interface/cli/commands/cognitive.py ✅ 5 commands (state, delete, affinity, handoff, insights) + 3 rich formatters
4 UCL UI page ui/app/cognitive/page.tsx ✅ Tab UI (Shared States + Affinity Scores) + StateCard + StateDetail + AffinityTable
5 Tests tests/unit/interface/test_cognitive_*.py ✅ 30 tests (19 API + 11 CLI), 1558 total unit tests

Sprint 7.6: Integration Testing + Benchmarks ✅ COMPLETE (2026-03-11)

Cross-engine scenarios, context continuity measurement, benchmarks.

# Item File Notes
1 Cross-engine integration tests tests/integration/test_ucl_cross_engine.py ✅ 13 tests (6 classes: handoff pipeline, adapter fidelity, insight roundtrip, affinity, conflict, benchmarks)
2 Context continuity benchmark benchmarks/context_continuity.py ✅ 97.2% overall (target >85%). AdapterScore + ContinuityResult + run_benchmark()
3 Memory deduplication accuracy benchmarks/dedup_accuracy.py ✅ 57.1% overall (target >50%). 3 scenarios + DedupScore + DedupResult
4 A2A protocol (if needed) ⏭️ Skipped — UCL already provides cross-engine communication via ContextAdapters + HandoffTask
5 Benchmark dashboard ui/app/benchmarks/page.tsx ✅ API (3 endpoints) + CLI (3 commands) + UI page + 11 new unit tests

Phase 7 Completion Criteria: ✅ ALL MET

  • Shared task state persists across agent handoffs (decisions, artifacts, blockers) — Sprint 7.1 + 7.4
  • Shared memory accessible from all 6 agent engines — Sprint 7.2 (ContextAdapters)
  • Context adapters inject/extract for each engine type — Sprint 7.2 (6 adapters)
  • Insight extraction auto-runs after task execution — Sprint 7.3 (ExtractInsightsUseCase)
  • Agent affinity scoring influences routing decisions — Sprint 7.4 (select_with_affinity)
  • Cross-engine task handoff demonstrated (Agent A → Agent B with full context) — Sprint 7.4 (HandoffTaskUseCase)
  • UCL exposed through API + CLI + UI — Sprint 7.5
  • Context continuity score > 85% in benchmarks — Sprint 7.6 (97.2% achieved)

Post-Phase 7: Self-Evolution Loop Closure

Sprint 8.1: Close the Self-Evolution Loop ✅ COMPLETE (2026-03-12)

ExecuteTaskUseCase runs tasks but never recorded ExecutionRecords. Phase 6's evolution engine was fully built but received zero data. This sprint closes the loop.

# Item File Notes
1 Auto-recording in ExecuteTaskUseCase application/use_cases/execute_task.py execution_record_repo + default_model params, time.monotonic() duration, _safe_record_execution() fire-and-forget
2 Task type inference application/use_cases/execute_task.py _infer_task_type() — TopicExtractor → TaskType mapping (10 topics)
3 Container wiring interface/api/container.py ✅ Moved execution_record_repo creation before execute_task, passed repo + default_model
4 Tests tests/unit/application/test_execute_task.py ✅ 7 tests (success/failure/fallback recording, recording failure doesn't block, None backward compat, duration positive, task_type inference)

Result: 1599 unit tests + 50 integration, 0 failures, lint clean.


Sprint 8.2: Production Verification + E2E Smoke Tests ✅ COMPLETE (2026-03-14)

First production-level operational verification. Fixed a critical bug where LiteLLMGateway ignored .env OLLAMA_DEFAULT_MODEL setting, causing 500 errors. Added automated verification infrastructure covering all 42 API endpoints.

# Item File Notes
1 Fix settings-driven default model infrastructure/llm/litellm_gateway.py _default_free_model property — replaces 4 hardcoded MODEL_TIERS[FREE][0] references with settings.ollama_default_model
2 Update gateway tests tests/unit/infrastructure/test_litellm_gateway.py DEFAULT_OLLAMA"ollama/qwen3:8b", explicit ollama_default_model in fixture
3 Smoke test script scripts/smoke_test.sh ✅ 21 curl-based checks (health, tasks, models, cost, engines, marketplace, evolution, UCL, benchmarks, error handling), ~10s
4 E2E test suite tests/e2e/test_api_smoke.py ✅ 29 pytest/httpx tests — auto-skips when API is down. Includes Ollama-powered task execution (120s timeout) + plan→approve flow

Production verification results:

  • Full stack: PostgreSQL+pgvector + Redis + Neo4j + Ollama (qwen3:8b) + FastAPI (PG mode) + Next.js 16
  • Task execution: "フィボナッチ数列を計算する関数を書いて" → 4 subtasks, success rate 100%, cost $0.00
  • Smoke test: 21/21 passed
  • E2E tests: 29/29 passed
  • Unit tests: 1599/1599 passed
  • Lint: clean

Result: 1599 unit + 29 E2E + 50 integration tests, 0 failures, lint clean.


Phase 9: Intelligence Layer — "Knowledge → Action" (PLANNED)

Phase 1-8 built the infrastructure (DAG, LLM routing, LAEE, UCL, Self-Evolution). Phase 9 bridges the gap between infrastructure and intelligent behavior.

Core problem observed: A FizzBuzz task is decomposed into 5 logical algorithm steps instead of 1 "write and run the code" action. The LLM generates text descriptions but never executes code. Results are shown as raw JSON in the UI. Plans are not reviewed before execution. Infrastructure exists but intelligence is absent.

Phase 9 closes these gaps with 4 sprints.

Dependency Graph

Sprint 9.1 (Smart Decomposition)     ← standalone, highest priority
    ↓
Sprint 9.2 (LAEE Code Execution)     ← benefits from 9.1 but parallelizable
    ↓
Sprint 9.3 (UI Result Formatting)    ← depends on 9.2 (code/output fields)
    ↓
Sprint 9.4 (Interactive Planning)    ← standalone, can start anytime

Sprint 9.1: Smart Decomposition (Intent Analyzer v2) ✅ COMPLETE (2026-03-14) — TD-067

Problem: IntentAnalyzer always decomposes into 2-5 subtasks regardless of task complexity. Simple tasks like "FizzBuzz" get split into algorithm logic steps instead of a single actionable coding task.

Solution: Add task complexity assessment to IntentAnalyzer. Simple tasks → 1 subtask. Complex tasks → meaningful decomposition. Subtask descriptions become action-oriented ("Write and execute Python code for X") instead of declarative ("Output Fizz when divisible by 3").

# Item File Notes
1 Task complexity classifier domain/services/task_complexity.py Pure domain service: classify goal → simple / medium / complex
2 Improved decomposition prompt infrastructure/task_graph/intent_analyzer.py Complexity-aware prompt: simple → 1 subtask, complex → 2-5. Action-oriented descriptions
3 Single-subtask path infrastructure/task_graph/intent_analyzer.py Skip LLM decomposition for simple tasks — wrap goal directly as single subtask
4 Unit tests tests/unit/domain/test_task_complexity.py Complexity classifier coverage
5 Unit tests tests/unit/infrastructure/test_intent_analyzer.py Decomposition prompt + single-task path

Completion Criteria:

  • "FizzBuzzを" → 1 subtask: "FizzBuzzのPythonコードを書いて実行する"
  • "REST API with auth + DB + tests" → 3 subtasks (meaningful decomposition)
  • Existing tests pass (backward compatible)

Sprint 9.2: LAEE Code Execution Integration ✅ COMPLETE (2026-03-14) — TD-068

Problem: _execute_batch() in engine.py sends subtask description to LLM and stores the text response. Code is never actually executed. LAEE's 40+ tools exist but are not wired to task execution.

Solution: After LLM generates a response, detect executable code blocks. If found, run via LAEE shell_exec. Store both code and execution output. Add code and execution_output fields to SubTask.

# Item File Notes
1 Code block extractor infrastructure/task_graph/code_executor.py Extract code from LLM markdown responses (python, bash, etc.)
2 Safe code execution infrastructure/task_graph/code_executor.py Execute via LAEE shell_exec with timeout, capture stdout/stderr
3 SubTask schema extension domain/entities/task.py +code: str | None, +execution_output: str | None (backward compatible)
4 Engine integration infrastructure/task_graph/engine.py execute_one(): LLM response → extract code → execute → structured result
5 API schema update interface/api/schemas.py +code, +execution_output on SubTaskResponse
6 Execution prompt infrastructure/task_graph/engine.py System prompt instructs LLM to produce runnable code blocks
7 Unit tests tests/unit/infrastructure/test_code_executor.py Code extraction + mock execution
8 E2E verification tests/e2e/test_api_smoke.py Verify code field populated for coding tasks

Completion Criteria:

  • "FizzBuzzを" → subtask.code contains Python code, subtask.execution_output contains "1\n2\nFizz\n4\nBuzz\n..."
  • Non-coding tasks (summaries, Q&A) still work without code execution
  • Execution timeout prevents infinite loops (default 30s)
  • LAEE approval mode respected (confirm-destructive blocks dangerous code)

Sprint 9.3: UI Result Formatting ✅ COMPLETE (2026-03-14) — TD-069

Problem: TaskDetail.tsx renders st.result as raw text. When LLM returns JSON or code, it displays unformatted. TaskGraph nodes show truncated raw strings.

Solution: Parse subtask results and render structured views: syntax-highlighted code blocks, formatted execution output, clean error display.

# Item File Notes
1 Code block component ui/components/CodeBlock.tsx Syntax highlighting (highlight.js or shiki), copy button, language label
2 Execution result component ui/components/ExecutionResult.tsx Code + output + status in structured layout
3 TaskDetail refactor ui/components/TaskDetail.tsx Detect code/execution_output fields, render CodeBlock + ExecutionResult
4 TaskGraph node improvement ui/components/TaskGraph.tsx Show meaningful label (not truncated JSON), status icon + model badge
5 Result parser utility ui/lib/resultParser.ts Parse subtask result: detect JSON, code blocks, plain text

Completion Criteria:

  • Code results display with syntax highlighting
  • Execution output shows in terminal-style block (dark bg, monospace)
  • JSON results are pretty-printed, not raw strings
  • Non-code results display cleanly as plain text
  • Graph nodes show clean descriptions (no raw JSON)

Sprint 9.4: Interactive Planning as Default ✅ COMPLETE (2026-03-14) — TD-070

Problem: POST /api/tasks immediately creates and executes a task. The InteractivePlanUseCase exists but is never used in the default flow. Users get no chance to review or modify the plan before execution.

Solution: Make plan-first the default mode. POST /api/tasks returns a plan for review. Approve triggers execution. Simple tasks can auto-approve based on complexity assessment from Sprint 9.1.

# Item File Notes
1 Plan-first API flow interface/api/routes/tasks.py POST /api/tasks → create plan, return for review. POST /api/tasks/{id}/approve → execute
2 Auto-approve setting shared/config.py PLANNING_AUTO_APPROVE_SIMPLE=true — skip review for simple tasks
3 Plan review UI ui/app/tasks/ Plan display with approve/reject buttons, cost estimate, model allocation
4 Backward compat mode shared/config.py PLANNING_MODE=interactive | auto | disableddisabled keeps current behavior
5 Unit tests tests/unit/interface/test_tasks_planning.py Plan-first flow, auto-approve, backward compat

Completion Criteria:

  • Default flow: create task → show plan → user approves → execute
  • Simple tasks auto-approve (1 subtask, low risk)
  • PLANNING_MODE=disabled preserves current behavior (no breaking change)
  • Plan shows cost estimate, model allocation, risk assessment
  • UI shows plan review screen with approve/reject/edit

Sprint 9.5: Full-Stack Structured Logging ✅ COMPLETE (2026-03-14) — TD-071

Problem: Debugging across the stack is difficult. Backend has no centralized logging. Frontend has zero logging. API calls, LLM interactions, and WebSocket events are invisible.

Solution: Centralized shared/logging.py with structured format. Add logger to all key backend modules. Create client-side ui/lib/logger.ts with API call tracing.

# Item File Notes
1 Centralized logging config shared/logging.py setup_logging() with structured format, third-party noise suppression
2 Log level setting shared/config.py +log_level: str = "INFO"
3 LLM gateway logging infrastructure/llm/litellm_gateway.py Model, tokens, cost, fallback warnings
4 Task engine logging infrastructure/task_graph/engine.py Decomposition, subtask exec, retries, finalization
5 Intent analyzer logging infrastructure/task_graph/intent_analyzer.py Complexity classification, LLM decomposition
6 Code executor logging infrastructure/task_graph/code_executor.py Block extraction, execution, timeout/failure
7 API route logging interface/api/routes/tasks.py, plans.py Request lifecycle, planning mode
8 API/CLI startup logging interface/api/main.py, interface/cli/main.py setup_logging() at startup
9 Frontend logger ui/lib/logger.ts [Morphic] prefix, level filtering via env var
10 API call tracing ui/lib/api.ts Method/path, status, timing (ms), WebSocket lifecycle

Tests: All 1,680 existing tests pass. Lint clean.

Phase 9 Summary

Phase 9 COMPLETE: 5 sprints (9.1–9.5)
  - TaskComplexityClassifier: SIMPLE/MEDIUM/COMPLEX goal classification
  - CodeExecutor: extract + execute code blocks from LLM responses
  - UI result formatting: CodeBlock, ExecutionResult, resultParser
  - Interactive planning as default (3 modes: interactive/auto/disabled)
  - Full-stack structured logging (backend + frontend)
  - 1,680 unit tests, 0 failures, lint clean

Phase 9 Success Metrics

Metric Target Actual
Simple task decomposition accuracy 1 subtask for simple tasks, 2-5 for complex ✅ Achieved (27 classifier tests)
Code execution success rate > 80% for standard coding tasks ✅ 28 tests covering extract+execute
UI readability (no raw JSON) 0 raw JSON in task detail view ✅ CodeBlock+ExecutionResult components
Plan approval UX < 2 clicks from goal input to execution start ✅ Auto-approve for SIMPLE tasks
Backward compatibility All existing tests pass ✅ 1,680 tests, 0 failures
Logging coverage All key modules traced ✅ 8 backend modules + 2 frontend files

Risk Management

Risk Impact Probability Mitigation
LangGraph breaking changes High Low Thin wrapper isolates API
Ollama model quality insufficient Medium Medium Auto-fallback to API by task type
Neo4j Community limitations Low Low No issue at Phase 1 scale
Agent CLI (OpenHands etc.) API changes Medium Medium AgentEngine Protocol abstraction
Phase 1 exceeds 2 weeks High Medium Minimize UI scope, prioritize API + CLI
mem0 pgvector compatibility Low Low Can fallback to direct pgvector usage

Critical Path

Sprint 1.1 (Infra)
    → Sprint 1.2 (LLM Layer) ← TOP PRIORITY. Delays here cascade
        → Sprint 1.3 (Task Graph) ← Core. All of Phase 2-7 depends on this
            → Sprint 1.3b (LAEE) ← Partially parallelizable with 1.3
            → Sprint 1.4 (Context Eng.) ← Partially parallelizable with 1.3
            → Sprint 1.5 (Memory) ← Partially parallelizable with 1.3
                → Sprint 1.6 (API + UI) ← Requires 1.3-1.5 completion
                    → Sprint 1.7 (E2E Test)

Bottleneck: Sprint 1.2 (LLM Layer) and Sprint 1.3 (Task Graph Engine) are most critical. Allocate maximum time here.


Success Metrics by Phase

Phase Metric Target
1 Task completion at $0 Yes
1 Ollama inference latency < 10s
2 Parallel execution speedup 3x+
2 Interactive Planning approval rate > 80%
2 CLI commands functional task, model, cost
3 Memory compression ratio 98% (10K→500 tokens)
3 Context restoration accuracy > 90%
4 Agent CLI routing accuracy > 85%
5 Memory compression ratio 98% (10K→500 tokens)
5 MCP tool safety scoring accuracy > 90%
6 Self-evolution strategy improvement +15%/month
7 Cross-engine context continuity > 85%
7 Memory deduplication accuracy > 90%
7 Task handoff success rate > 90%
9 Simple task → 1 subtask rate > 90%
9 Code execution success rate > 80%
9 Raw JSON in UI 0 instances
9 Plan-to-execution clicks < 2

Phase 4: Agent CLI Orchestration — Sprint Detail

Sprint 4.1: AgentEngine Domain Foundation (COMPLETE — 2026-02-27)

Deliverables: Domain-only foundation for two-tier routing.

# Item Status
1 AgentEngineType value object (6 engines) DONE
2 TaskType extended (+LONG_RUNNING_DEV, +WORKFLOW_PIPELINE) DONE
3 AgentEnginePort ABC + AgentEngineResult + AgentEngineCapabilities DONE
4 AgentEngineRouter domain service (pure static, 3 methods) DONE
5 infrastructure/agent_cli/ package stub DONE
6 __init__.py exports updated DONE

Tests: 46 new (10 port + 36 router), 729 total unit tests, 0 failures. Lint: ruff check 0 errors, ruff format clean.

Completion Criteria: All 46 tests pass. No regressions on existing 683 tests. Lint clean.

Sprint 4.2: Engine Drivers (COMPLETE — 2026-02-27)

Deliverables: 5 concrete AgentEnginePort implementations + SubprocessMixin + container wiring.

# File Description
1 infrastructure/agent_cli/_subprocess_base.py CLIResult + SubprocessMixin
2 infrastructure/agent_cli/ollama_driver.py Wraps LiteLLMGateway + OllamaManager
3 infrastructure/agent_cli/claude_code_driver.py claude -p --output-format json
4 infrastructure/agent_cli/codex_cli_driver.py codex exec --json --full-auto
5 infrastructure/agent_cli/gemini_cli_driver.py gemini -p --output-format json
6 infrastructure/agent_cli/openhands_driver.py httpx REST: POST create + GET poll

Modified: shared/config.py (+4 fields), infrastructure/agent_cli/__init__.py (re-exports), interface/api/container.py (+_wire_agent_drivers)

Tests: 92 new (13+16+16+16+18+13 per driver), total 821 unit tests. Lint clean.

ADK deferred: Requires google-adk pip dep. Router fallback chain handles: ADK -> GEMINI_CLI -> CLAUDE_CODE -> OLLAMA.

Sprint 4.3: RouteToEngine Use Case + API/CLI Integration (COMPLETE — 2026-03-02)

Deliverables: RouteToEngineUseCase (fallback chain execution) + 3 API endpoints + 2 CLI commands + formatters.

# File Description
1 application/use_cases/route_to_engine.py RouteToEngineUseCase: list_engines, get_engine, execute (fallback chain)
2 interface/api/routes/engines.py GET /api/engines, GET /api/engines/{type}, POST /api/engines/run
3 interface/api/schemas.py +EngineRunRequest, EngineInfoResponse, EngineListResponse, EngineRunResponse
4 interface/cli/commands/engine.py morphic engine {list, run} with --engine, --type, --budget flags
5 interface/cli/formatters.py +print_engine_table(), print_engine_result()

Modified: interface/api/container.py (+RouteToEngineUseCase DI wiring), interface/api/main.py (+engines_router), interface/cli/main.py (+engine_app)

Tests: 43 new (23 use case + 12 API + 8 CLI), total 864 unit tests, 0 failures. Lint: ruff check 0 errors, ruff format clean.

Key design: RouteToEngineUseCase._build_chain() builds ordered engine list via AgentEngineRouter.select_with_fallbacks(), then iterates: is_available() → run_task() → return on first success. Preferred engine override prepended to chain. All engines fail → returns last error.

| 5 | Auto tool discovery success rate | > 60% | | 6 | Monthly improvement rate | +15% | | 7 | SWE-bench lite score | TBD |

Sprint 4.4: Agent Engine Integration Tests (COMPLETE — 2026-03-03)

Deliverables: 30 integration tests across 9 test classes verifying Phase 4 completion criteria.

# File Description
1 tests/integration/test_agent_engines.py 30 tests, 9 classes — live engine execution + routing + fallback

Test Classes:

Class Tests Purpose
TestOllamaEngineLive 3 Baseline: task execution + zero cost + availability
TestClaudeCodeEngineLive 3 Headless CLI: task + capabilities + availability
TestCodexCLIEngineLive 3 Codex exec: task + capabilities + availability
TestGeminiCLIEngineLive 3 Gemini CLI: task + capabilities + availability
TestOpenHandsEngineLive 3 REST API: task + capabilities + availability
TestAvailabilityDetection 5 All 5 engines: is_available() accuracy
TestCrossEngineLive 3 Criteria 1: same task cross-engine comparison
TestRoutingLive 5 Criteria 2,3: routing + fallback verification
TestDisabledDriverBehavior 2 Disabled driver returns proper failure

Design: Follows test_cloud_llm.py pattern — module-scoped fixtures, graceful skip on env/auth errors (nested Claude session, expired tokens). Engines with CLI on PATH but runtime issues are detected via _is_env_error() helper.

Results: 18 passed, 12 skipped (Ollama offline + nested/auth skips), 0 failed. Total unit+integration: 864+18 = 882 passing tests.

Completion Criteria Met:

  • Same task across multiple engines with result comparison (TestCrossEngineLive)
  • Task-type-based automatic engine selection (TestRoutingLive: budget=0→OLLAMA, SIMPLE_QA→OLLAMA, COMPLEX_REASONING→chain)
  • Availability check + fallback in live environment (TestRoutingLive: fallback_when_primary_unavailable)

Sprint 4.5: ADK Driver (COMPLETE — 2026-03-05)

Deliverables: Google ADK (Agent Development Kit) driver via Python SDK with try-import guard.

# File Description
1 infrastructure/agent_cli/adk_driver.py ADKDriver: LlmAgent→Runner→run_async, try-import guard
2 pyproject.toml +adk = ["google-adk>=1.0"] optional dep
3 shared/config.py +adk_enabled, adk_default_model settings
4 interface/api/container.py +ADK wiring in _wire_agent_drivers()
5 tests/unit/infrastructure/test_adk_driver.py 17 tests (4 classes)
6 tests/integration/test_agent_engines.py +TestADKEngineLive (3 tests) + availability

Tests: 17 new unit, total 881 unit tests. Integration: +7 tests (ADK live + availability + count update).

Key design: Optional dependency guard (_ADK_AVAILABLE sentinel) — same pattern as neo4j/pgvector. Module-level try-import assigns None to SDK classes when missing. Disabled/not-installed → error result without raising.

Sprint 4.6: Knowledge File Management (COMPLETE — 2026-03-05)

Deliverables: Engine-specific context injection at the use case layer.

# File Description
1 infrastructure/agent_cli/knowledge_loader.py KnowledgeFileLoader: engine→filename mapping, read from project root
2 application/use_cases/route_to_engine.py +`context: str
3 interface/api/schemas.py +context field on EngineRunRequest
4 interface/api/routes/engines.py Pass context=body.context to execute()
5 interface/cli/commands/engine.py +--context / -c flag on run command
6 tests/unit/infrastructure/test_knowledge_loader.py 13 tests (3 classes)
7 tests/unit/application/test_route_to_engine.py +4 context injection tests

Tests: 17 new (13 loader + 4 context), total 898 unit tests. Lint clean.

Key design: Context injected at use case layer (not driver) — backward compatible. No ABC changes. KnowledgeFileLoader maps engine type to conventional filename (CLAUDE.md, AGENTS.md, llms-full.txt). format_context() combines knowledge file + optional extra context.

Phase 4 Summary

Phase 4 COMPLETE: 6 sprints (4.1–4.6)
  - 6 AgentEnginePort drivers (Ollama, ClaudeCode, Codex, Gemini, OpenHands, ADK)
  - AgentEngineRouter domain service (pure static, task→engine mapping)
  - RouteToEngineUseCase (fallback chain + context injection)
  - KnowledgeFileLoader (engine-specific project context)
  - 3 API endpoints + 2 CLI commands
  - 898 unit tests + 37 integration tests = 935 total

Phase 10: UX Intelligence — Prompt Quality + UI Redesign

Sprint 10.1: Complexity-Aware Execution Prompts (TD-072)

Goal: LLM output quality matches task complexity.

New files:

  • domain/value_objects/execution_config.py — frozen dataclass
  • domain/services/execution_prompt_builder.py — complexity → prompt/params
  • domain/services/answer_extractor.py — strip <think>, preambles
  • tests/unit/domain/test_execution_prompt_builder.py — 13 tests
  • tests/unit/domain/test_answer_extractor.py — 12 tests

Modified files:

  • domain/entities/task.pySubTask.complexity field
  • infrastructure/task_graph/intent_analyzer.py — stamp complexity
  • infrastructure/task_graph/engine.py — use builder + extractor
  • interface/api/schemas.pycomplexity in SubTaskResponse
  • ui/lib/api.tscomplexity in SubTask interface

Sprint 10.2: React Flow Node Redesign (TD-073)

Goal: Proper DAG visualization with readable nodes.

New files:

  • ui/lib/graphLayout.tscomputeNodeDepths(), computeGraphHeight(), truncateWithEllipsis()

Modified files:

  • ui/components/TaskGraph.tsx — depth-based layout, smoothstep edges, dynamic height, tooltips

Sprint 10.3: Result Display Restructure (TD-074)

Goal: Separate answer from reasoning, complexity-aware rendering.

Modified files:

  • ui/lib/resultParser.ts — added parseResultWithComplexity()
  • ui/components/TaskDetail.tsx — SIMPLE → large answer, MEDIUM/COMPLEX → structured + collapsible reasoning

Phase 10 Summary

Phase 10 COMPLETE: 3 sprints (10.1–10.3)
  - ExecutionPromptBuilder + AnswerExtractor (pure domain services)
  - Complexity-aware LLM params (temp 0.15–0.7, tokens 512–4096)
  - DAG-based graph layout with depth positioning
  - Complexity-aware result display (answer/reasoning separation)
  - 25 new tests, total 1705 unit tests, 0 failures, lint clean
  - TD-072, TD-073, TD-074

Phase 11: Intelligence Quality — Semantic Dedup + Hybrid Classifier

Sprint 11.1: Semantic Dedup in InsightExtractor (TD-075)

Goal: Dedup accuracy 57.1% → 90%+ via embedding-based similarity.

Modified files:

  • infrastructure/cognitive/insight_extractor.py_semantic_dedup(), EmbeddingPort injection, MemoryClassifierPort injection
  • shared/config.pysemantic_dedup_enabled, semantic_dedup_threshold
  • interface/api/container.py — DI wiring for embedding + classifier
  • tests/unit/infrastructure/test_mcp_server.py_FakeSettings updated

New files:

  • tests/unit/infrastructure/test_insight_extractor_semantic_dedup.py — 9 tests

Sprint 11.2: HybridMemoryClassifier (TD-076)

Goal: Regex-first + LLM fallback for text classification.

New files:

  • domain/ports/memory_classifier.pyMemoryClassifierPort ABC
  • infrastructure/cognitive/hybrid_memory_classifier.py — regex + LLM hybrid
  • tests/unit/domain/test_memory_classifier_port.py — 3 tests
  • tests/unit/infrastructure/test_hybrid_memory_classifier.py — 20 tests

Sprint 11.3: Benchmark + Docs Update

Goal: Add paraphrased_facts scenario, update ADRs and docs.

Modified files:

  • benchmarks/dedup_accuracy.pyparaphrased_facts scenario + embedding_port param
  • docs/TECH_DECISIONS.md — TD-075, TD-076
  • docs/IMPLEMENTATION_PLAN.md — Phase 11
  • docs/ARCHITECTURE.md — component counts

Phase 11 Summary

Phase 11 COMPLETE: 3 sprints (11.1–11.3)
  - Semantic dedup via EmbeddingPort + cosine similarity (threshold 0.85)
  - MemoryClassifierPort ABC + HybridMemoryClassifier (regex + LLM)
  - Graceful degradation: embedding fail → exact-match, LLM fail → regex
  - Paraphrased facts benchmark scenario
  - 32 new tests, total 1737 unit tests, 0 failures, lint clean
  - TD-075, TD-076

Phase 12: ReAct Loop + Model-Aware Decomposition

Sprint 12.1: ReAct Loop Core (TD-077)

Goal: Implement Think → Act → Observe iterative loop for tool-augmented execution.

New files:

  • domain/entities/react_trace.py — ReactTrace, ReactStep, ToolCallRecord
  • domain/services/react_controller.py — should_continue(), build_tool_result_message()
  • infrastructure/task_graph/react_executor.py — ReactExecutor (max 10 iterations, 3000 char observation truncation)

Modified files:

  • domain/entities/task.py — +tool_calls_count, +react_iterations on SubTask
  • domain/ports/llm_gateway.py — +complete_with_tools(), +ToolCallResult

Sprint 12.2: Tool Schema Registry (TD-078)

Goal: OpenAI-compatible JSON schemas for all 38 LAEE tools.

New files:

  • domain/entities/tool_schema.py — ToolSchema, ParameterProperty, to_openai_tool()
  • domain/services/tool_selector.py — TaskType → tool profile mapping
  • infrastructure/local_execution/tools/tool_schemas.py — 38 tool definitions

Sprint 12.3: Web Search + Fetch Tools (TD-079)

Goal: API-key-free web access for information retrieval.

New files:

  • infrastructure/local_execution/tools/web_tools.py — web_search (DuckDuckGo HTML), web_fetch (HTTP + strip)

Sprint 12.4: LiteLLM Tool Calling Integration (TD-080)

Goal: Production tool-calling support across all LLM providers.

Modified files:

  • infrastructure/llm/litellm_gateway.py — complete_with_tools() implementation
  • infrastructure/task_graph/engine.py — ReAct integration into LangGraphTaskEngine
  • interface/api/container.py — DI wiring for ReactExecutor

Sprint 12.5: Model-Aware Subtask Decomposition (TD-081)

Goal: Extract model preferences from natural language, route subtasks to specific models.

New files:

  • domain/value_objects/model_preference.py — ModelPreference (models, clean_goal)
  • domain/services/model_preference_extractor.py — regex-based model alias extraction (EN+JP)
  • tests/unit/domain/test_model_preference_extractor.py

Modified files:

  • domain/entities/task.py — +preferred_model on SubTask
  • infrastructure/task_graph/intent_analyzer.py — multi-model → per-model subtasks
  • infrastructure/task_graph/engine.py — preferred_model routing

Sprint 12.6: Dynamic Multi-Model Decomposition via LLM (TD-082)

Goal: Replace static goal-copy with LLM-driven differentiated decomposition.

New files:

  • domain/value_objects/collaboration_mode.py — CollaborationMode (PARALLEL/COMPARISON/DIVERSE/AUTO)
  • domain/services/model_capability_registry.py — model-id → capability description
  • tests/unit/domain/test_model_capability_registry.py

Modified files:

  • domain/value_objects/model_preference.py — +collaboration_mode field
  • domain/services/model_preference_extractor.py — +collaboration mode detection (regex, priority-based)
  • infrastructure/task_graph/intent_analyzer.py — +_llm_multi_model_decompose(), +_parse_multi_model_response(), +mode guidance, +static fallback
  • domain/value_objects/__init__.py — +CollaborationMode export
  • domain/services/__init__.py — +ModelCapabilityRegistry export
  • tests/unit/domain/test_model_preference_extractor.py — +14 collaboration mode tests
  • tests/unit/infrastructure/test_intent_analyzer.py — updated multi-model tests + 3 new tests

Sprint 12.7: Two Worlds Integration (TD-083)

Goal: Connect execution pipeline to full infrastructure — per-engine routing, tools tracking, MCP integration, auto-upgrade, discussion phase, DEGRADED validation.

New files:

  • tests/unit/domain/test_degraded_status.py — 9 tests
  • tests/unit/domain/test_requires_tools.py — 17 tests
  • tests/unit/infrastructure/test_react_tools_tracking.py — 5 tests
  • tests/unit/infrastructure/test_engine_two_worlds.py — 10 tests

Modified files:

  • domain/value_objects/status.py — +DEGRADED status
  • domain/entities/task.py — +tools_used, data_sources, DEGRADED handling
  • domain/services/task_complexity.py — +requires_tools()
  • infrastructure/task_graph/react_executor.py — +tools tracking, +MCP routing
  • infrastructure/task_graph/engine.py — +per-engine routing, +auto-upgrade, +discussion, +DEGRADED validation
  • interface/api/container.py — +MCP wiring, +route_to_engine injection
  • interface/api/schemas.py — +tools_used, data_sources in SubTaskResponse

Phase 12 Summary

Phase 12 COMPLETE: 7 sprints (12.1–12.7)
  - ReAct iterative tool-calling loop (10 max iterations, observation truncation)
  - 38 LAEE tool schemas in OpenAI-compatible format
  - Web search (DuckDuckGo) + web fetch (HTML strip) — zero API keys
  - LiteLLM tool-calling across providers (Claude, GPT, Gemini, Ollama)
  - Model preference extraction from natural language (EN + JP)
  - CollaborationMode detection (COMPARISON > DIVERSE > PARALLEL > AUTO)
  - LLM-based multi-model decomposition with capability-aware prompts
  - Two Worlds integration: per-engine routing, auto-upgrade, discussion phase, DEGRADED validation, MCP routing, tools tracking
  - Graceful fallbacks: LLM fail → static decomposition, embedding fail → exact-match, engine fail → ReAct
  - 170 new tests, total 1923 unit tests, 0 failures, lint clean
  - TD-077, TD-078, TD-079, TD-080, TD-081, TD-082, TD-083