|
| 1 | +{"_type":"issue","id":"robot-jf3","title":"Mute button — mute/unmute robot microphone from dashboard","description":"Add a mute/unmute button to the dashboard so the user can be sure the robot isn't listening.\n\nOptions for placement:\n- Fixed in the nav bar (always visible, like a quick control)\n- On the settings page\n- Both: a small mic icon in the nav + full control in settings\n\nImplementation approach:\n- Write a mute state file (e.g., data/mute.flag) that the agent checks\n- Or use pipewire/ALSA to mute the capture device directly: wpctl set-mute @DEFAULT_SOURCE@ toggle\n- The agent's AudioRecorder should check mute state and skip recording when muted\n- Visual indicator: red mic-off icon when muted, green mic icon when live\n\nThe mute should be RELIABLE — the user needs to trust that when muted, nothing is being heard or transcribed.","status":"open","priority":1,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-27T00:25:16Z","created_by":"David Birks","updated_at":"2026-04-27T00:25:16Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
1 | 2 | {"_type":"issue","id":"robot-fjb","title":"Barge-in detection — allow interrupting robot while it speaks","description":"Allow users to interrupt the robot mid-speech, while ignoring background noise.\n\n## Key finding: XMOS XVF3800 has built-in AEC\nThe hardware echo cancellation already removes the robot's own voice from the mic signal. Any speech detected during playback is genuinely from the environment. This makes barge-in detection much simpler.\n\n## Architecture:\n1. Replace blocking sd.play()+sd.wait() with non-blocking sd.OutputStream callback\n2. Monitor mic via VAD while audio plays (main thread, not a new thread)\n3. Classify interrupt: duration gate (480ms sustained speech) + DOA (front hemisphere)\n4. On confirmed barge-in: stop playback, capture remaining speech, transcribe, process as new turn\n\n## Implementation plan:\n1. New app/interruptible_player.py — OutputStream callback + VAD monitoring\n2. Modify audio_io.py — share VAD model, add record_from_buffer()\n3. Modify agent_client.py — add inject_interruption() for partial context\n4. Modify orchestrator.py — replace play calls with interruptible player\n5. Modify main.py — wire up shared VAD and player\n\n## Classification (cheapest first):\n- Duration gate: 480ms sustained VAD (filters coughs, brief sounds)\n- DOA gate: front hemisphere only (filters side conversations)\n- Energy gate: minimum RMS threshold\n- NOT using semantic classifier (too slow for barge-in, 250-550ms)\n\n## What happens on interrupt:\n- Stop OutputStream immediately\n- Reset wobbler\n- Capture user's speech (already buffered from VAD monitoring)\n- Track partial response for LLM context\n- Process as new conversation turn\n\n## Estimated: ~300 lines across 3-4 files","status":"closed","priority":1,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-26T19:55:34Z","created_by":"David Birks","updated_at":"2026-04-26T20:06:51Z","started_at":"2026-04-26T20:04:08Z","closed_at":"2026-04-26T20:06:51Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
2 | 3 | {"_type":"issue","id":"robot-9xa","title":"CPU spike causes audio to stop — face tracker or movement manager spinning","description":"After running for a while (~30-60 min), the agent process spikes to 200%+ CPU and stops processing audio. The VAD/STT pipeline freezes because the CPU is saturated.\n\n## Symptoms:\n- Process shows 200-280% CPU usage\n- No new STT/User: log entries\n- Process is still running, not crashed\n- Restart fixes it temporarily\n\n## Likely cause:\n- InsightFace face detection thread runs continuously with no frame rate limit\n- At 640x640 det_size on i7-6700K, each detection takes ~200-300ms\n- The detection loop has no sleep — it grabs and processes frames as fast as possible\n- Over time this may starve the audio recording thread of CPU\n\n## Possible fixes:\n- Add a sleep/rate limit to the face detection loop (e.g., cap at 5 FPS)\n- Lower det_size to 320x320 (faster detection, less CPU)\n- Add a frame skip when CPU load is high\n- Only run face detection when not in active conversation","status":"closed","priority":1,"issue_type":"bug","owner":"david@birks.dev","created_at":"2026-04-26T19:21:58Z","created_by":"David Birks","updated_at":"2026-04-26T19:22:57Z","closed_at":"2026-04-26T19:22:57Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
3 | 4 | {"_type":"issue","id":"robot-ox3","title":"Natural gaze behavior — look at speaker, look away while thinking, look back when responding","description":"The robot should have natural conversational gaze behavior:\n\n1. **While user is speaking**: Look at the speaker (DOA direction) and hold gaze steadily — no spazzing or looking away\n2. **While thinking (LLM processing)**: Gently look away a few degrees to the side — mimics human \"thinking\" behavior \n3. **While responding (TTS playing)**: Look back at the speaker\n\nThis is the most natural human conversational pattern. Currently the robot spazzes/looks away erratically during listening due to DOA noise or face tracking jitter.\n\n## Current behavior issues:\n- DOA tracking + face tracking can fight each other during conversation\n- The thinking animation includes yaw drift which looks like unfocused looking-around rather than deliberate thinking\n- No coordination between gaze direction and conversation phase (listening/thinking/speaking)\n\n## Proposed implementation:\n- During LISTENING: lock gaze toward DOA direction (or face if visible). Use the spring system but with a longer halflife to prevent jitter. Ignore small DOA changes (apply dead zone to DOA too).\n- During THINKING: offset gaze 5-10 degrees to the side from last speaker position. Smooth transition. Keep the subtle thinking animation but smaller/calmer.\n- During SPEAKING: smoothly return gaze to speaker direction. Hold steady while TTS plays.\n\n## Relates to:\n- robot-tl0 (smooth motion — closed, spring system in place)\n- robot-b68 (gaze aversion — this is a more specific version of that)\n- DOA tracker already provides speaker direction\n- MovementManager already has listening/processing state flags\n\n## Key insight from research:\n- Humans look at speakers ~60% of the time with 2.2s average fixation\n- Looking away while thinking is a universal human behavior\n- The transition timing matters: look away should be slow/gentle, look back should be slightly faster","status":"closed","priority":1,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-26T02:31:42Z","created_by":"David Birks","updated_at":"2026-04-26T02:39:00Z","started_at":"2026-04-26T02:35:58Z","closed_at":"2026-04-26T02:39:00Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
|
11 | 12 | {"_type":"issue","id":"robot-9os","title":"Qwen3-TTS voice cloning — British male voice","description":"Set up voice cloning with Qwen3-TTS Base model to create a consistent British male voice for the robot.\n\n## Current state:\n- Qwen3-TTS GGUF (1.7B CustomVoice) is set up at /home/david/dev/Qwen3-TTS-GGUF\n- Running on GPU via llama.cpp, RTF ~1.2\n- Currently using Piper (fast but robotic) while voice is being designed\n- VoiceDesign demo available for auditioning voices\n- No community British male voice presets exist\n\n## Plan:\n1. Use VoiceDesign HuggingFace Space to design ideal British male voice\n2. Download the generated audio as reference clip\n3. Set up Base model GGUF export (same HaujetZhao pipeline)\n4. Clone voice from reference clip, save as voice.json\n5. Use cloned voice for all runtime synthesis\n\n## Key info:\n- Base model is same 1.7B architecture, same VRAM footprint\n- Voice cloning needs 10-15s clean reference audio + transcript\n- Accent may drift toward American on longer outputs (known limitation)\n- seed pinning keeps voice consistent across generations","status":"open","priority":2,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T02:15:51Z","created_by":"David Birks","updated_at":"2026-04-26T02:15:51Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
12 | 13 | {"_type":"issue","id":"robot-vr8","title":"TTS engine selection — British male voice options","description":"Research on TTS engines with built-in British male voices that fit GTX 1070 (8GB).\n\n## Only two viable options:\n\n**Kokoro (recommended):** 4 British male voices (bm_george, bm_daniel, bm_fable, bm_lewis). 82M params, CPU, ~2-3s/sentence. Top quality. kokoro-onnx variant may be faster.\n\n**Piper (current):** 1 British male (northern_english_male-medium) + alan + VCTK multi-speaker. CPU, sub-second. Lower quality.\n\n## Everything else fails hardware/voice constraints:\n- Voxtral: SM 6.1 incompatible, too large\n- Parler-TTS: 30s+ on CPU, too slow\n- MeloTTS: British voice is female only\n- XTTS v2: Clone only, no built-in British\n- Edge-TTS: Cloud only\n- Qwen3-TTS: No built-in British, needs voice cloning\n\n## Future path:\n- Try kokoro-onnx for faster CPU inference\n- Qwen3-TTS voice cloning when ready to design a voice\n- Revisit when upgrading GPU","status":"open","priority":3,"issue_type":"feature","owner":"david@birks.dev","created_at":"2026-04-26T19:20:46Z","created_by":"David Birks","updated_at":"2026-04-26T19:20:46Z","dependency_count":0,"dependent_count":0,"comment_count":0} |
13 | 14 | {"_type":"issue","id":"robot-b68","title":"Gaze aversion — natural look-away behavior","description":"Add human-like gaze aversion so the robot doesn't stare continuously at faces.\n\n## Research findings:\n- Humans look at speakers ~60% of the time, with 2.2s average fixation\n- Mutual eye contact only 3.5% of conversation time\n- Robot should look away every 2-5s for 0.5-2s, then return\n- Look-away should be subtle (10-15 degrees down or to the side)\n- Pollen/Kuri use similar patterns for natural-feeling interaction\n\n## Implementation:\n- Add gaze hold timer (2-5s random) in MovementManager\n- During look-away, shift face tracking offset slightly\n- Return gaze slightly faster than departure (asymmetric)\n- Disable during active conversation (when robot is speaking or listening)\n\nDepends on: robot-tl0 (smooth motion) — gaze aversion needs the spring system to look natural","status":"closed","priority":3,"issue_type":"feature","assignee":"David Birks","owner":"david@birks.dev","created_at":"2026-04-26T02:16:03Z","created_by":"David Birks","updated_at":"2026-04-26T02:51:14Z","started_at":"2026-04-26T02:47:00Z","closed_at":"2026-04-26T02:51:14Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} |
14 | | -{"_type":"memory","key":"when-user-asks-to-add-a-beads-issue","value":"When user asks to add a beads issue: 1) Create a stub issue with bd create to track it immediately. 2) Usually kick off a background agent to research the topic thoroughly — existing codebase, online best practices, HuggingFace/Pollen demos, how to integrate with existing code. 3) The research should be documented in the issue description or comments. 4) This is not just a toy — aim for production quality that serves as a platform for ROS/LLM experimentation. 5) Always investigate Pollen demo repos and Reachy Mini SDK patterns."} |
15 | 15 | {"_type":"memory","key":"for-ad-hoc-browser-testing-of-the-dashboard","value":"For ad-hoc browser testing of the dashboard, use: pnpx @anthropic-ai/playwright-mcp@latest --url http://dave-droid:3001 — lets you visually inspect the page without a full test suite"} |
| 16 | +{"_type":"memory","key":"when-user-asks-to-add-a-beads-issue","value":"When user asks to add a beads issue: 1) Create a stub issue with bd create to track it immediately. 2) Usually kick off a background agent to research the topic thoroughly — existing codebase, online best practices, HuggingFace/Pollen demos, how to integrate with existing code. 3) The research should be documented in the issue description or comments. 4) This is not just a toy — aim for production quality that serves as a platform for ROS/LLM experimentation. 5) Always investigate Pollen demo repos and Reachy Mini SDK patterns."} |
0 commit comments