feat: Recordings — save, play, re-transcribe, delete (with format + storage cap) - #60
Open
o2scale wants to merge 24 commits into
Open
feat: Recordings — save, play, re-transcribe, delete (with format + storage cap)#60o2scale wants to merge 24 commits into
o2scale wants to merge 24 commits into
Conversation
Persist each recording's audio to disk when enabled, with Tauri commands for the Recordings page (Lane B frontend). - AppConfig: save_recordings (bool, default false) + recording_format (String, default "flac"; wav|flac|mp3), mirroring the capsule_auto_hide pattern. - audio/encode.rs: RecordingFormat + encode_pcm(); WAV reuses build_wav, FLAC via flacenc (pure Rust), MP3 via mp3lame-encoder behind the encode-mp3 feature (in default). - storage: idempotent recording_file column migration + HistoryEntry field; find_by_id, set/clear_recording_file, list_recordings, update_raw_text. - pipeline: tee PCM (provider-agnostic) when enabled; on audio close, encode and write <app_data_dir>/recordings/<UTC-ts>.<ext> and store the path on the row. - stt: shared post_audio upload + transcribe_encoded; retranscribe_file re-runs the configured Whisper-compatible provider on the saved file. - commands/recordings.rs: RecordingEntry + get_recordings / get_recording_path / retranscribe_recording / delete_recording, registered in lib.rs. - tauri.conf.json: enable asset protocol (+ protocol-asset feature) with $APPDATA/recordings scope and CSP media-src/img-src for asset playback. Tests: 133 pass (default + --no-default-features); clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YZ5THh48cPsz572DTJiRqM
…gant spring pop-up
# Conflicts: # tasks/lessons.md # tasks/todo.md
…ecordings cap Completes the Recordings slice and fixes three defects surfaced during the live dual-leg verify (record/list/play/re-transcribe/delete all pass). Playback - WebKitGTK rejects Tauri's custom asset:// scheme as an <audio> media source (fetch works, playback fails with MEDIA_ERR_SRC_NOT_SUPPORTED). Add a read_recording_bytes command that returns the file as raw IPC bytes; the UI plays from a blob: URL with an explicit MIME instead of convertFileSrc. - New RecordingPlayer component: raw MP3 streams carry no duration header so WebKit reports Infinity and breaks the native seekbar. Use the backend's stored duration_ms for the timeline and force a one-time end-seek so the element becomes seekable. On-brand play/pause + scrubber. Re-transcribe persistence - update_raw_text -> update_transcript: overwrite both raw_text and polished_text. The list renders polished_text || raw_text, so leaving a stale polished_text reverted the new transcript on reload. Saved-recordings cap - New max_saved_recordings config (u32, 0 = unlimited, default 0). After each saved recording the pipeline prunes the oldest audio beyond the cap via prune_recordings_over(), clearing recording_file while keeping the transcript (mirrors manual delete). Settings number field (0-999) + i18n + config patch. UX - Recordings list shows the recording id (#N). - Capsule anchors top-center (8px margin), lingers ~3s on the complete state, and drops clipped outer shadows that caused edge glitches. Tests: 135 Rust + 125 frontend unit tests passing; production build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3ge7fdVvWkmEmbqttcvGr
Lift the practical cap on a single recording from 5 min to ~1 h: - Settings slider max 300s -> 3600s (frontend auto-stop in DurationTimer) - STT buffer cap is now provider-aware: local servers (no API key required) get 256 MB (~2.2 h); cloud Whisper APIs keep 24 MB (OpenAI/Groq 25 MB upload limit) so BYOK providers do not silently break - STT finalize timeout 120s -> 600s so long local transcriptions are not cut off while the server is still processing Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
When recording-to-disk is enabled, the audio file was written on the audio channel close regardless of STT outcome, but stop() returned early on STT failure / no-speech without calling save_history. That left the MP3 on disk with no DB row — invisible in the Recordings page, not re-transcribable, and ignored by the prune cap (orphaned file). Now stop()'s None branch records a history entry (empty transcript) whenever a recording file was actually written, then enforces the prune cap. Aborted sessions write no file (recording_file is None), so they still save nothing. Result: a failed transcription leaves a visible, re-transcribable recording instead of an orphan — which is exactly when keeping the audio matters most. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Owner
|
Review update: I would not merge this PR as-is yet. The feature direction is useful, but the current implementation still has correctness/safety risks that should be fixed before landing:
The local issue-remediation branch I just validated does not merge this PR. Recommendation: keep this PR open and fix the above items, or split Recordings into a smaller safer PR. |
Three related improvements after diagnosing transcription failures under CPU load: - STT timeout (whisper_compat): for local providers the request timeout now scales with clip length (~4x duration, 2-30 min) instead of a flat 60s, and a local timeout no longer retries — so a slow-under-load transcription waits and succeeds instead of timing out and re-sending the clip (the pile-up spiral that left recordings untranscribed). Cloud keeps 60s + retries. - Crash-proof recording (pipeline + lib): PCM streams to a `.pcm.partial` file while recording, removed on a successful save. Any leftover partial (from an abort or crash) is recovered on startup: encoded to the configured format, added to history with an empty transcript, and surfaced in Recordings for re-transcription. Closes the window where aborted/crashed audio was lost. - Capsule communication (frontend): live "Transcribing... Ns / server busy" status so it never looks frozen; clearer failure messages plus a "Recording saved - re-transcribe" hint when recordings are on; and a "Failed - re-transcribe" marker in the Recordings list for empty-transcript entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Two Home/Recordings issues: - Total Recordings stat was capped at 200. HomePage rendered `history.length`, but history is loaded with `getHistory(200, 0)` — so the stat maxed out at the page size instead of the true total. Add a `get_history_count` command backed by `SELECT COUNT(*)`, store it, and display the real count (refreshed on each pipeline completion alongside the history list). - Recordings play button often did nothing. The list eagerly loaded every recording's audio into a blob and mounted a live <audio> element for each, so ~100 simultaneous WebKitGTK/GStreamer pipelines were created — far past what the backend sustains, and most never became playable while `play()` rejections were silently swallowed. Load audio lazily on first play of a row (auto-start once bytes arrive) and log play/element errors instead of swallowing them, so only the recordings actually played open a media pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Recording playback fetches audio bytes over IPC and plays them from a `URL.createObjectURL` blob URL (the asset:// scheme is rejected by WebKitGTK as a media source). But the CSP `media-src` directive omitted `blob:`, so the webview refused every blob URL and no recording could play. Add `blob:` to media-src. Pairs with the lazy-load change so playback is both permitted and resource-safe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
…ation Two capsule UX issues on GNOME Wayland: - The capsule appeared in the upper-left corner instead of top-center. Wayland does not let a client position its own top-level window, so the existing "re-anchor to top-center" setPosition calls were silently no-ops and the compositor placed the capsule wherever it liked. Force the GTK/WebKit backend onto XWayland (GDK_BACKEND=x11) on Wayland sessions when an X server is reachable, where absolute positioning works; the top-center math then takes effect. Also nudge the top gap from 8 to 14 px. - The "completed" confirmation flashed for only ~66 ms. The backend returns to idle almost immediately after `outputting`, unmounting CapsuleComplete before its self-timer could run. Add a transient `justCompleted` flag (set on `outputting`, cleared after 2.5 s) that keeps the confirmation — now labelled "Transcribed" — visible for a couple of seconds as a clear success signal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Add an audible confirmation when a transcription is delivered. The freedesktop "bell" sound is converted to a small WAV and embedded in the binary (no runtime file or external player), then played via rodio (built on the cpal already in use) on a detached, best-effort thread from output_text() right after output succeeds. Playback waits a bounded ~700ms rather than sleep_until_end so a misbehaving output device can never block or leak the thread; any audio error is logged at debug and ignored. A gated smoke test (OT_AUDIO_TEST=1) exercises it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Recording playback flipped to "playing" but produced no sound and never advanced. WebKitGTK's media backend loads a blob: URL but cannot preroll the GStreamer pipeline from it, so the element reports playing while nothing decodes. Resolve each recording's audio lazily through Tauri's asset protocol instead (getRecordingPath + convertFileSrc -> http://asset.localhost/...), which serves the file over HTTP with range support that the media backend needs. The CSP already allows http://asset.localhost in media-src and the asset scope already covers the recordings dir. Drops the blob-URL machinery; keeps lazy-on-first-play so only the recordings actually played open a pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Recordings entered "playing" state but produced no sound and sat at 0:00. On load the player seeked the element to currentTime=1e7 to coax WebKit into computing a duration for the seekbar. But the saved files are raw MP3 (ADTS, no Xing/LAME header), so the element's duration stays Infinity, the durationchange rewind-to-0 never fires, and the playhead is left at the end of the stream — so play() starts at EOF: onPlay fires but nothing advances and there is no audio. GStreamer decodes these files fine, confirming the file/codec are good and the fault was this hack. Drop the seek entirely and rely on the backend durationMs (already authoritative) for the timeline. Also surface the media error code in the UI so playback failures are visible, not silent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
The asset-protocol attempt produced MEDIA_ERR_SRC_NOT_SUPPORTED (err 4): WebKitGTK's <audio> only accepts http/file/blob sources and rejects Tauri's asset:// scheme as a media source. On Linux convertFileSrc yields asset:// (the http://asset.localhost form is Windows-only), so that approach can't work here. Go back to fetching bytes over IPC and playing from a blob: URL (allowed by the CSP). Crucially this is now paired with the removed duration-seek hack — blob-without-the-hack was never actually tested, since the hack was dropped only after the asset switch. Lazy-on-first-play is kept to avoid mounting ~100 media pipelines at once. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
…t/blob) Recording playback failed with MEDIA_ERR_SRC_NOT_SUPPORTED (err 4) for every source we tried. Root cause (confirmed against tauri-apps/tauri #8654, #9326, #9573, #3725 and verified directly against webkit2gtk-4.1 2.52.3): on Linux, WebKitGTK plays media via GStreamer, which has no URI handler for Tauri's asset:// scheme, and blob: media is also broken inside the Tauri webview. Both fail with err 4. http:// sources, however, play fine — proven with a scripted WebKitGTK instance (cross-origin http://127.0.0.1 media with a media-src 'self' http://127.0.0.1:* CSP reaches readyState=4). So serve recordings over a tiny localhost http server (tiny_http) on a random 127.0.0.1 port, with Content-Type and HTTP Range support so seeking works, and point the <audio> element at http://127.0.0.1:<port>/<filename>. The frontend fetches the port via get_recordings_server_port and drops all blob/IPC-bytes machinery; preload="none" means a pipeline only opens on play. CSP gains http://127.0.0.1:* in media-src. Filenames are validated (no path traversal). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SBiTZ7qez7USTDnZSf243F
Tauri's global-shortcut grab (X11 XGrabKey) never receives keys on GNOME/Wayland, where the compositor owns the keyboard. Add a toggle_recording() helper and route a `--toggle-recording` CLI arg through the single-instance plugin so a desktop-environment shortcut (e.g. a GNOME custom keybinding) can drive start/stop recording without stealing focus. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add local_stt_status/load/unload commands that proxy to the local
custom-whisper server's /health and /v1/local/{load,unload} endpoints,
and a LocalModelControl panel in the Speech Recognition settings (shown
only for localhost base URLs) to view VRAM state and load/offload the
model on demand. Pairs with the server's lazy-load support so freeing
the GPU doesn't stop the STT service.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mpt) In clipboard mode the app simulated Ctrl+V via enigo after copying. On Wayland that goes through the RemoteDesktop portal, which pops an "Allow Remote Interaction" permission dialog on every paste. Skip the simulated paste on Wayland and leave the transcript on the clipboard for a manual Ctrl+V, matching how keyboard output is already disabled there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 4GB T1200 is a hard wall on Linux (no VRAM paging), so a game on the card leaves no room for Whisper's encode workspace and dictation fails. The server now falls back to CPU on OOM; this exposes the choice in-app. - local_stt_set_device command -> POST /v1/local/device - Auto/GPU/CPU selector in the local model panel, with an amber state and a live cooldown note while transcription is on the CPU - Load/Offload disabled under a CPU pin (the model isn't in VRAM) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qVgUx4EVM5j5Znbc7sLJk
Formatting only, no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qVgUx4EVM5j5Znbc7sLJk
otl-toggle.sh and run-dev.sh hold machine-specific paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qVgUx4EVM5j5Znbc7sLJk
Recordings had re-transcribe and delete but no way to copy the transcript, unlike History. Adds a copy action alongside them. - Copies polished_text, falling back to raw_text - Disabled when the entry has no text (failed transcriptions) - Shows a transient "Copied" indicator, mirroring History - Localised across all 10 locales, reusing each language's existing history.copied / history.failedToCopy wording for consistency Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qVgUx4EVM5j5Znbc7sLJk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Adds a complete Recordings feature: every dictation can be saved to disk and managed from a dedicated Recordings screen — play, re-transcribe, and delete — with a user-selectable audio format and a configurable storage cap.
Features
0 = unlimited) — the oldest audio is auto-pruned past the cap (transcripts kept), to bound disk usage.Implementation notes
recording_file/duration_mscolumns onhistory;list_recordings,find_by_id,set/clear_recording_file,update_transcript, andprune_recordings_over.flacenc(pure Rust, always on),mp3lame-encoderbehind a defaultencode-mp3feature, WAV via the existing Whisper PCM path. The capture PCM is tee'd in the STT loop and written to<app_data>/recordings/<UTC>.<ext>.blob:URLs, not the asset protocol. WebKitGTK's<audio>rejects Tauri's customasset://scheme as a media source (fetchworks, playback fails withMEDIA_ERR_SRC_NOT_SUPPORTED). Aread_recording_bytescommand returns the file as raw IPC bytes; the UI builds a blob with an explicit MIME. This is also correct cross-platform.Infinityand breaks scrubbing.RecordingPlayeruses the backend's storedduration_msfor the timeline and forces a one-time end-seek so the element becomes seekable.update_transcriptwrites bothraw_textandpolished_text; the list renderspolished_text || raw_text, so updating only the raw column reverted the new transcript on reload.Test plan
update_transcript, andprune_recordings_over(cap + unlimited).tsc+ production build green.🤖 Generated with Claude Code
https://claude.ai/code/session_01X3ge7fdVvWkmEmbqttcvGr