Feat/library stems midi first class#30
Merged
Merged
Conversation
Adds a scrcpy-protocol relay sidecar (sidecars/questcast, @yume-chan ADB + AdbScrcpyOptions3_3_3) that pipes raw H.264 over a WebSocket, plus the backend questcast module that supervises it and exposes status/devices with a verbose log ring buffer for field debugging. The browser side decodes via WebCodecs into an offscreen canvas and captureStream()s it as a VJ camera source. The key fix for the "keyframes decode but nothing renders" symptom: scrcpy sends SPS/PPS in a separate config packet, so those bytes are prepended to every keyframe chunk before feeding the decoder. VJView drops the floating double-decode preview (the right-panel preview reuses the single stream) and gains clipboard-write on the iframe allow so the in-VJ log copy button works. core/adb.py centralises adb path resolution shared by questcast/questmidi.
Phase 1 backend of the watch-link feature: a tiny signaling relay so viewers
can watch the live VJ output (with audio) over a shareable link. Media stays
peer-to-peer — only SDP/ICE pass through the relay — so the venue LAN gets
full quality and low latency.
GET /api/broadcast/link room id + LAN/public watch URLs + ICE config
GET /api/broadcast/watch/{room} self-contained WebRTC viewer page
WS /api/broadcast/ws rooms, routes offer/answer/ice by viewer id
STUN always; optional TURN from theDAW_TURN_URL/USER/PASS env for public
reach. Public base from theDAW_PUBLIC_BASE. The VJ-side GO LIVE broadcaster
(captureStream of the output canvas + audio) is the next phase.
Replaces the typed export path with a real OS folder dialog. core/folder_dialog opens a Windows FolderBrowserDialog (PowerShell, -STA, topmost) with a tkinter fallback. The vj module gains GET /export-folder and POST /export-folder/pick, the latter persisting the chosen vj.export_root via the settings patch.
Three field-incident fixes:
- Pin setuptools<81 so basic-pitch's legacy `pkg_resources` import keeps
working; setuptools >= 81 removed it and broke MIDI conversion with
"No module named 'pkg_resources'".
- midi/engine.py re-exposes librosa.core.audio.{to_mono,resample,util} as a
compat shim for piano_transcription_inference, which librosa 0.11 moved.
- questmidi/bridge.py resolves adb through the shared core.adb helper.
.gitignore now ignores node_modules/ everywhere (frontend + sidecars) so package trees never get vendored. Appends section 9 to the global-layout plan capturing what landed (Quest streaming, layout, cymatics, dep fixes, broadcast phase 1) and the queued VJ UI punchlist.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[feat(questcast): stream Quest headset video into the VJ as a live source]
Adds a scrcpy-protocol relay sidecar (sidecars/questcast, @yume-chan ADB +
AdbScrcpyOptions3_3_3) that pipes raw H.264 over a WebSocket, plus the
backend questcast module that supervises it and exposes status/devices with
a verbose log ring buffer for field debugging.
The browser side decodes via WebCodecs into an offscreen canvas and
captureStream()s it as a VJ camera source. The key fix for the
"keyframes decode but nothing renders" symptom: scrcpy sends SPS/PPS in a
separate config packet, so those bytes are prepended to every keyframe
chunk before feeding the decoder.
VJView drops the floating double-decode preview (the right-panel preview
reuses the single stream) and gains clipboard-write on the iframe allow so
the in-VJ log copy button works.
core/adb.py centralises adb path resolution shared by questcast/questmidi.
[feat(broadcast): WebRTC watch-link signaling for the live VJ output]
Phase 1 backend of the watch-link feature: a tiny signaling relay so viewers
can watch the live VJ output (with audio) over a shareable link. Media stays
peer-to-peer — only SDP/ICE pass through the relay — so the venue LAN gets
full quality and low latency.
GET /api/broadcast/link room id + LAN/public watch URLs + ICE config
GET /api/broadcast/watch/{room} self-contained WebRTC viewer page
WS /api/broadcast/ws rooms, routes offer/answer/ice by viewer id
STUN always; optional TURN from theDAW_TURN_URL/USER/PASS env for public
reach. Public base from theDAW_PUBLIC_BASE. The VJ-side GO LIVE broadcaster
(captureStream of the output canvas + audio) is the next phase.
[feat(vj): native OS folder picker for VJ export output]
Replaces the typed export path with a real OS folder dialog. core/folder_dialog
opens a Windows FolderBrowserDialog (PowerShell, -STA, topmost) with a tkinter
fallback. The vj module gains GET /export-folder and POST /export-folder/pick,
the latter persisting the chosen vj.export_root via the settings patch.
[fix(backend): repair MIDI/basic-pitch deps and Quest adb resolution]
Three field-incident fixes:
pkg_resourcesimport keepsworking; setuptools >= 81 removed it and broke MIDI conversion with
"No module named 'pkg_resources'".
compat shim for piano_transcription_inference, which librosa 0.11 moved.
[chore: ignore node_modules globally; record VJ work + queued punchlist]
.gitignore now ignores node_modules/ everywhere (frontend + sidecars) so
package trees never get vendored. Appends section 9 to the global-layout plan
capturing what landed (Quest streaming, layout, cymatics, dep fixes, broadcast
phase 1) and the queued VJ UI punchlist