Skip to content

Feat/library stems midi first class#30

Merged
danieljtrujillo merged 5 commits into
mainfrom
feat/library-stems-midi-first-class
Jun 14, 2026
Merged

Feat/library stems midi first class#30
danieljtrujillo merged 5 commits into
mainfrom
feat/library-stems-midi-first-class

Conversation

@danieljtrujillo

Copy link
Copy Markdown
Collaborator

[feat(questcast): stream Quest headset video into the VJ as a live source]

Adds a scrcpy-protocol relay sidecar (sidecars/questcast, @yume-chan ADB +
AdbScrcpyOptions3_3_3) that pipes raw H.264 over a WebSocket, plus the
backend questcast module that supervises it and exposes status/devices with
a verbose log ring buffer for field debugging.

The browser side decodes via WebCodecs into an offscreen canvas and
captureStream()s it as a VJ camera source. The key fix for the
"keyframes decode but nothing renders" symptom: scrcpy sends SPS/PPS in a
separate config packet, so those bytes are prepended to every keyframe
chunk before feeding the decoder.

VJView drops the floating double-decode preview (the right-panel preview
reuses the single stream) and gains clipboard-write on the iframe allow so
the in-VJ log copy button works.

core/adb.py centralises adb path resolution shared by questcast/questmidi.

[feat(broadcast): WebRTC watch-link signaling for the live VJ output]

Phase 1 backend of the watch-link feature: a tiny signaling relay so viewers
can watch the live VJ output (with audio) over a shareable link. Media stays
peer-to-peer — only SDP/ICE pass through the relay — so the venue LAN gets
full quality and low latency.

GET /api/broadcast/link room id + LAN/public watch URLs + ICE config
GET /api/broadcast/watch/{room} self-contained WebRTC viewer page
WS /api/broadcast/ws rooms, routes offer/answer/ice by viewer id

STUN always; optional TURN from theDAW_TURN_URL/USER/PASS env for public
reach. Public base from theDAW_PUBLIC_BASE. The VJ-side GO LIVE broadcaster
(captureStream of the output canvas + audio) is the next phase.

[feat(vj): native OS folder picker for VJ export output]

Replaces the typed export path with a real OS folder dialog. core/folder_dialog
opens a Windows FolderBrowserDialog (PowerShell, -STA, topmost) with a tkinter
fallback. The vj module gains GET /export-folder and POST /export-folder/pick,
the latter persisting the chosen vj.export_root via the settings patch.

[fix(backend): repair MIDI/basic-pitch deps and Quest adb resolution]

Three field-incident fixes:

  • Pin setuptools<81 so basic-pitch's legacy pkg_resources import keeps
    working; setuptools >= 81 removed it and broke MIDI conversion with
    "No module named 'pkg_resources'".
  • midi/engine.py re-exposes librosa.core.audio.{to_mono,resample,util} as a
    compat shim for piano_transcription_inference, which librosa 0.11 moved.
  • questmidi/bridge.py resolves adb through the shared core.adb helper.

[chore: ignore node_modules globally; record VJ work + queued punchlist]

.gitignore now ignores node_modules/ everywhere (frontend + sidecars) so
package trees never get vendored. Appends section 9 to the global-layout plan
capturing what landed (Quest streaming, layout, cymatics, dep fixes, broadcast
phase 1) and the queued VJ UI punchlist

Adds a scrcpy-protocol relay sidecar (sidecars/questcast, @yume-chan ADB +
AdbScrcpyOptions3_3_3) that pipes raw H.264 over a WebSocket, plus the
backend questcast module that supervises it and exposes status/devices with
a verbose log ring buffer for field debugging.

The browser side decodes via WebCodecs into an offscreen canvas and
captureStream()s it as a VJ camera source. The key fix for the
"keyframes decode but nothing renders" symptom: scrcpy sends SPS/PPS in a
separate config packet, so those bytes are prepended to every keyframe
chunk before feeding the decoder.

VJView drops the floating double-decode preview (the right-panel preview
reuses the single stream) and gains clipboard-write on the iframe allow so
the in-VJ log copy button works.

core/adb.py centralises adb path resolution shared by questcast/questmidi.
Phase 1 backend of the watch-link feature: a tiny signaling relay so viewers
can watch the live VJ output (with audio) over a shareable link. Media stays
peer-to-peer — only SDP/ICE pass through the relay — so the venue LAN gets
full quality and low latency.

  GET  /api/broadcast/link         room id + LAN/public watch URLs + ICE config
  GET  /api/broadcast/watch/{room} self-contained WebRTC viewer page
  WS   /api/broadcast/ws           rooms, routes offer/answer/ice by viewer id

STUN always; optional TURN from theDAW_TURN_URL/USER/PASS env for public
reach. Public base from theDAW_PUBLIC_BASE. The VJ-side GO LIVE broadcaster
(captureStream of the output canvas + audio) is the next phase.
Replaces the typed export path with a real OS folder dialog. core/folder_dialog
opens a Windows FolderBrowserDialog (PowerShell, -STA, topmost) with a tkinter
fallback. The vj module gains GET /export-folder and POST /export-folder/pick,
the latter persisting the chosen vj.export_root via the settings patch.
Three field-incident fixes:

- Pin setuptools<81 so basic-pitch's legacy `pkg_resources` import keeps
  working; setuptools >= 81 removed it and broke MIDI conversion with
  "No module named 'pkg_resources'".
- midi/engine.py re-exposes librosa.core.audio.{to_mono,resample,util} as a
  compat shim for piano_transcription_inference, which librosa 0.11 moved.
- questmidi/bridge.py resolves adb through the shared core.adb helper.
.gitignore now ignores node_modules/ everywhere (frontend + sidecars) so
package trees never get vendored. Appends section 9 to the global-layout plan
capturing what landed (Quest streaming, layout, cymatics, dep fixes, broadcast
phase 1) and the queued VJ UI punchlist.
@danieljtrujillo danieljtrujillo merged commit 29a2702 into main Jun 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant