|
1 | 1 | {"_type":"issue","id":"robot-7e6","title":"XMOS XVF3800 mic goes silent — requires USB replug to recover","description":"XMOS XVF3800 mic goes silent — SOLUTION FOUND\n\n## Root cause: USB autosuspend + XMOS firmware bug\nThe kernel suspends the USB capture endpoint while playback stays active. The XMOS firmware doesn't recover properly from this.\n\n## Fixes applied:\n1. **udev rule** — disabled USB autosuspend for the XMOS device (preventive)\n2. **XMOS REBOOT command** — confirmed working! Sends a vendor control transfer that reboots the entire DSP. Mic recovers from software, no physical unplug needed.\n\n## Still needed:\n- Watchdog thread that monitors mic RMS and auto-sends REBOOT when silence detected\n- Test if the autosuspend fix alone prevents the issue","status":"open","priority":0,"issue_type":"bug","owner":"david@birks.dev","created_at":"2026-04-28T22:30:26Z","created_by":"David Birks","updated_at":"2026-04-28T22:43:36Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
2 | | -{"_type":"issue","id":"robot-nxq","title":"Robot gets stuck processing backlog — needs to discard stale audio","description":"When the robot doesn't respond for a while (e.g., during a long tool call or crash recovery), it accumulates a backlog of VAD-triggered audio. Instead of processing it all sequentially, it should discard stale audio and only process the most recent utterance. The conversation loop should have a 'freshness' check — if the audio was recorded more than N seconds ago, skip it.","status":"open","priority":1,"issue_type":"bug","owner":"david@birks.dev","created_at":"2026-04-28T23:18:43Z","created_by":"David Birks","updated_at":"2026-04-28T23:18:43Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
| 2 | +{"_type":"issue","id":"robot-nxq","title":"Robot gets stuck processing backlog — needs to discard stale audio","description":"When the robot doesn't respond for a while (e.g., during a long tool call or crash recovery), it accumulates a backlog of VAD-triggered audio. Instead of processing it all sequentially, it should discard stale audio and only process the most recent utterance. The conversation loop should have a 'freshness' check — if the audio was recorded more than N seconds ago, skip it.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-28T23:18:43Z","created_by":"David Birks","updated_at":"2026-04-28T23:24:29Z","started_at":"2026-04-28T23:24:29Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
3 | 3 | {"_type":"issue","id":"robot-6wr","title":"XMOS mic watchdog — auto-recover when capture goes silent","description":"Build a watchdog that monitors mic RMS and sends the XMOS REBOOT command when silence is detected. The REBOOT command is confirmed working via respeaker.write('REBOOT', [1]). Also document the autosuspend fix and XMOS findings in CLAUDE.md.","status":"closed","priority":1,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-28T22:50:36Z","created_by":"David Birks","updated_at":"2026-04-28T22:53:40Z","started_at":"2026-04-28T22:51:16Z","closed_at":"2026-04-28T22:53:40Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
4 | | -{"_type":"issue","id":"robot-0sb","title":"STT quality — research better speech-to-text options","description":"Current faster-whisper medium.en has accuracy issues, especially when not speaking directly at the robot. Research alternatives including GPU models, CPU streaming models, and whether GPT-4o-mini-transcribe style models exist locally.","status":"open","priority":1,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-28T22:50:35Z","created_by":"David Birks","updated_at":"2026-04-28T22:50:35Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
| 4 | +{"_type":"issue","id":"robot-0sb","title":"STT quality — research better speech-to-text options","description":"Current faster-whisper medium.en has accuracy issues, especially when not speaking directly at the robot. Research alternatives including GPU models, CPU streaming models, and whether GPT-4o-mini-transcribe style models exist locally.","status":"closed","priority":1,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-28T22:50:35Z","created_by":"David Birks","updated_at":"2026-04-28T23:20:01Z","closed_at":"2026-04-28T23:20:01Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
5 | 5 | {"_type":"issue","id":"robot-jf3","title":"Mute button — mute/unmute robot microphone from dashboard","description":"Add a mute/unmute button to the dashboard so the user can be sure the robot isn't listening.\n\nOptions for placement:\n- Fixed in the nav bar (always visible, like a quick control)\n- On the settings page\n- Both: a small mic icon in the nav + full control in settings\n\nImplementation approach:\n- Write a mute state file (e.g., data/mute.flag) that the agent checks\n- Or use pipewire/ALSA to mute the capture device directly: wpctl set-mute @DEFAULT_SOURCE@ toggle\n- The agent's AudioRecorder should check mute state and skip recording when muted\n- Visual indicator: red mic-off icon when muted, green mic icon when live\n\nThe mute should be RELIABLE — the user needs to trust that when muted, nothing is being heard or transcribed.","status":"closed","priority":1,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-27T00:25:16Z","created_by":"David Birks","updated_at":"2026-04-27T01:16:27Z","closed_at":"2026-04-27T01:16:27Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
6 | 6 | {"_type":"issue","id":"robot-fjb","title":"Barge-in detection — allow interrupting robot while it speaks","description":"Allow users to interrupt the robot mid-speech, while ignoring background noise.\n\n## Key finding: XMOS XVF3800 has built-in AEC\nThe hardware echo cancellation already removes the robot's own voice from the mic signal. Any speech detected during playback is genuinely from the environment. This makes barge-in detection much simpler.\n\n## Architecture:\n1. Replace blocking sd.play()+sd.wait() with non-blocking sd.OutputStream callback\n2. Monitor mic via VAD while audio plays (main thread, not a new thread)\n3. Classify interrupt: duration gate (480ms sustained speech) + DOA (front hemisphere)\n4. On confirmed barge-in: stop playback, capture remaining speech, transcribe, process as new turn\n\n## Implementation plan:\n1. New app/interruptible_player.py — OutputStream callback + VAD monitoring\n2. Modify audio_io.py — share VAD model, add record_from_buffer()\n3. Modify agent_client.py — add inject_interruption() for partial context\n4. Modify orchestrator.py — replace play calls with interruptible player\n5. Modify main.py — wire up shared VAD and player\n\n## Classification (cheapest first):\n- Duration gate: 480ms sustained VAD (filters coughs, brief sounds)\n- DOA gate: front hemisphere only (filters side conversations)\n- Energy gate: minimum RMS threshold\n- NOT using semantic classifier (too slow for barge-in, 250-550ms)\n\n## What happens on interrupt:\n- Stop OutputStream immediately\n- Reset wobbler\n- Capture user's speech (already buffered from VAD monitoring)\n- Track partial response for LLM context\n- Process as new conversation turn\n\n## Estimated: ~300 lines across 3-4 files","status":"closed","priority":1,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-26T19:55:34Z","created_by":"David Birks","updated_at":"2026-04-26T20:06:51Z","started_at":"2026-04-26T20:04:08Z","closed_at":"2026-04-26T20:06:51Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
7 | 7 | {"_type":"issue","id":"robot-9xa","title":"CPU spike causes audio to stop — face tracker or movement manager spinning","description":"After running for a while (~30-60 min), the agent process spikes to 200%+ CPU and stops processing audio. The VAD/STT pipeline freezes because the CPU is saturated.\n\n## Symptoms:\n- Process shows 200-280% CPU usage\n- No new STT/User: log entries\n- Process is still running, not crashed\n- Restart fixes it temporarily\n\n## Likely cause:\n- InsightFace face detection thread runs continuously with no frame rate limit\n- At 640x640 det_size on i7-6700K, each detection takes ~200-300ms\n- The detection loop has no sleep — it grabs and processes frames as fast as possible\n- Over time this may starve the audio recording thread of CPU\n\n## Possible fixes:\n- Add a sleep/rate limit to the face detection loop (e.g., cap at 5 FPS)\n- Lower det_size to 320x320 (faster detection, less CPU)\n- Add a frame skip when CPU load is high\n- Only run face detection when not in active conversation","status":"closed","priority":1,"issue_type":"bug","owner":"david@birks.dev","created_at":"2026-04-26T19:21:58Z","created_by":"David Birks","updated_at":"2026-04-26T19:22:57Z","closed_at":"2026-04-26T19:22:57Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
|
16 | 16 | {"_type":"issue","id":"robot-jaf","title":"Memory system — short-term and long-term with people profiles","description":"Research and implement a proper memory system for the robot.\n\n## Architecture (from research)\n\nFour tiers, all in SQLite:\n\n**Tier 1: Core Memory (~200 tokens, always in context)**\n- Who is present (from face recognition), their key facts, active topic\n- Refreshed on person change or topic shift\n\n**Tier 2: People Profiles (retrieved on face recognition)**\n- Linked to face embeddings via person_id\n- Structured: name, first_seen, last_seen, interaction_count, relationship\n- Unstructured facts: preferences, personal details, conversation topics\n\n**Tier 3: Semantic Memory (retrieved on demand)**\n- General facts with 384-dim embeddings (all-MiniLM-L6-v2, already loaded for wake detection)\n- Hybrid search: FTS5 keyword + cosine similarity\n\n**Tier 4: Episodic Memory (existing session_store.py)**\n- Full conversation history, already working\n\n## Tools: remember, recall, forget\n- remember: save fact with optional person link and category\n- recall: hybrid search (FTS5 + embedding similarity)\n- forget: remove a specific fact\n\n## Implementation phases:\n1. memory_store.py — SQLite schema + hybrid search\n2. People profiles — link face_tracker to memory\n3. Smart context injection — rebuild system prompt with relevant memories\n4. Replace flat memory.md with structured tools\n5. Share all-MiniLM-L6-v2 between WakeDetector and MemoryStore\n\n## Token budget (8K context):\n- System prompt: ~200 tokens\n- Core memory (person + facts): ~200 tokens\n- Retrieved memories: ~200 tokens\n- Conversation (20 messages): ~6,000 tokens\n- Tool schemas: ~800 tokens\n- Response: ~600 tokens\n\n## Key decisions:\n- No knowledge graph (overkill for \u003c1K memories)\n- No separate extraction LLM call (let LLM use remember tool)\n- No sqlite-vec (numpy cosine is fast enough for hundreds of vectors)\n- Share embedding model instance between wake detector and memory","status":"open","priority":2,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T02:18:52Z","created_by":"David Birks","updated_at":"2026-04-26T02:23:26Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
17 | 17 | {"_type":"issue","id":"robot-rjq","title":"Streaming TTS for Qwen3-TTS GGUF","description":"When switching back to Qwen3-TTS, implement sentence-level streaming to reduce time-to-first-audio.\n\n## Current state:\n- LLM streaming is implemented (send_streaming yields sentences)\n- Piper gets TTS'd per-sentence (near-instant, works great)\n- Qwen3-TTS GGUF has built-in streaming support (TTSConfig.streaming=True)\n- The TTS server currently waits for full synthesis before returning\n\n## Plan:\n- Qwen3-TTS engine's SpeakerWorker already handles streaming playback internally\n- Option A: Add /synthesize_stream endpoint with chunked transfer encoding\n- Option B: Load Qwen3-TTS engine in-process (bypass HTTP overhead)\n- GPU contention: LLM and TTS talker share GPU, natural serialization per-sentence\n\n## Expected improvement:\n- Time-to-first-audio with Qwen3-TTS: ~1.5-3s (currently ~5-6s)","status":"open","priority":2,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T02:16:15Z","created_by":"David Birks","updated_at":"2026-04-26T02:16:15Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
18 | 18 | {"_type":"issue","id":"robot-9os","title":"Qwen3-TTS voice cloning — British male voice","description":"Set up voice cloning with Qwen3-TTS Base model to create a consistent British male voice for the robot.\n\n## Current state:\n- Qwen3-TTS GGUF (1.7B CustomVoice) is set up at /home/david/dev/Qwen3-TTS-GGUF\n- Running on GPU via llama.cpp, RTF ~1.2\n- Currently using Piper (fast but robotic) while voice is being designed\n- VoiceDesign demo available for auditioning voices\n- No community British male voice presets exist\n\n## Plan:\n1. Use VoiceDesign HuggingFace Space to design ideal British male voice\n2. Download the generated audio as reference clip\n3. Set up Base model GGUF export (same HaujetZhao pipeline)\n4. Clone voice from reference clip, save as voice.json\n5. Use cloned voice for all runtime synthesis\n\n## Key info:\n- Base model is same 1.7B architecture, same VRAM footprint\n- Voice cloning needs 10-15s clean reference audio + transcript\n- Accent may drift toward American on longer outputs (known limitation)\n- seed pinning keeps voice consistent across generations","status":"open","priority":2,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T02:15:51Z","created_by":"David Birks","updated_at":"2026-04-26T02:15:51Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
| 19 | +{"_type":"issue","id":"robot-gb6","title":"Robot personality controls — humor/sarcasm level like Interstellar's TARS","description":"Research how TARS/CASE from Interstellar handle adjustable personality settings (humor at 75%, honesty at 90%, etc.). Explore how to give the Reachy Mini similar personality knobs that can be adjusted via the dashboard.","status":"open","priority":3,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-28T23:24:28Z","created_by":"David Birks","updated_at":"2026-04-28T23:24:28Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
19 | 20 | {"_type":"issue","id":"robot-vr8","title":"TTS engine selection — British male voice options","description":"Research on TTS engines with built-in British male voices that fit GTX 1070 (8GB).\n\n## Only two viable options:\n\n**Kokoro (recommended):** 4 British male voices (bm_george, bm_daniel, bm_fable, bm_lewis). 82M params, CPU, ~2-3s/sentence. Top quality. kokoro-onnx variant may be faster.\n\n**Piper (current):** 1 British male (northern_english_male-medium) + alan + VCTK multi-speaker. CPU, sub-second. Lower quality.\n\n## Everything else fails hardware/voice constraints:\n- Voxtral: SM 6.1 incompatible, too large\n- Parler-TTS: 30s+ on CPU, too slow\n- MeloTTS: British voice is female only\n- XTTS v2: Clone only, no built-in British\n- Edge-TTS: Cloud only\n- Qwen3-TTS: No built-in British, needs voice cloning\n\n## Future path:\n- Try kokoro-onnx for faster CPU inference\n- Qwen3-TTS voice cloning when ready to design a voice\n- Revisit when upgrading GPU","status":"open","priority":3,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T19:20:46Z","created_by":"David Birks","updated_at":"2026-04-26T19:20:46Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
20 | 21 | {"_type":"issue","id":"robot-b68","title":"Gaze aversion — natural look-away behavior","description":"Add human-like gaze aversion so the robot doesn't stare continuously at faces.\n\n## Research findings:\n- Humans look at speakers ~60% of the time, with 2.2s average fixation\n- Mutual eye contact only 3.5% of conversation time\n- Robot should look away every 2-5s for 0.5-2s, then return\n- Look-away should be subtle (10-15 degrees down or to the side)\n- Pollen/Kuri use similar patterns for natural-feeling interaction\n\n## Implementation:\n- Add gaze hold timer (2-5s random) in MovementManager\n- During look-away, shift face tracking offset slightly\n- Return gaze slightly faster than departure (asymmetric)\n- Disable during active conversation (when robot is speaking or listening)\n\nDepends on: robot-tl0 (smooth motion) — gaze aversion needs the spring system to look natural","status":"closed","priority":3,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-26T02:16:03Z","created_by":"David Birks","updated_at":"2026-04-26T02:51:14Z","started_at":"2026-04-26T02:47:00Z","closed_at":"2026-04-26T02:51:14Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
21 | 22 | {"_type":"memory","key":"for-ad-hoc-browser-testing-of-the-dashboard","value":"For ad-hoc browser testing of the dashboard, use: pnpx @anthropic-ai/playwright-mcp@latest --url http://dave-droid:3001 — lets you visually inspect the page without a full test suite"} |
|
0 commit comments