Fix FrameProcessor lifecycle for selector based noise cancellation#5433
Open
Topherhindman wants to merge 4 commits intomainfrom
Open
Fix FrameProcessor lifecycle for selector based noise cancellation#5433Topherhindman wants to merge 4 commits intomainfrom
Topherhindman wants to merge 4 commits intomainfrom
Conversation
When noise_cancellation is a NoiseCancellationSelector (callable) that returns a FrameProcessor, the framework never called _on_stream_info_updated or _on_credentials_updated on it. This caused plugins like ai_coustics to log "Missing configuration" on every frame and return audio unprocessed. The root cause: _create_stream() resolved the selector but only passed the result to AudioStream.from_track() without storing it as self._processor. The lifecycle calls in _on_track_available() check self._processor, which was None for the selector path. Store the selector-returned FrameProcessor as self._processor in _create_stream() so the existing lifecycle code can find it.
After _close_stream() closes a FrameProcessor, self._processor was left pointing at the closed object. This caused stale lifecycle calls in two scenarios: 1. A selector returns a FrameProcessor for one track but NoiseCancellationOptions for the next — the old closed processor still received _on_stream_info_updated and _on_credentials_updated calls. 2. A track is unpublished with no replacement (e.g., user switches microphones, client reconnects) — subsequent _on_token_refreshed calls hit the closed processor. Clear self._processor in _close_stream() after closing it, and in _create_stream() when a selector returns a non-FrameProcessor.
_ParticipantInputStream.aclose() closed the audio stream and unregistered room callbacks but never called _close() on an active FrameProcessor. This left processor cleanup dependent on garbage collection rather than deterministic teardown. This is a pre-existing gap that affects both direct and selector-backed FrameProcessors (e.g., ai_coustics, Krisp).
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the lifecycle for
FrameProcessor-based input noise cancellation whenAudioInputOptions.noise_cancellationis provided as aNoiseCancellationSelector.Before this change, if a selector returned a
FrameProcessor(for exampleai_coustics), the processor was passed tortc.AudioStream.from_track()but was not stored as the active input processor. As a result, it never received_on_stream_info_updated()or_on_credentials_updated(), which caused processors that depend on that configuration to run uninitialized.While fixing that, this PR also addresses two related lifecycle gaps:
Root cause
_ParticipantAudioInputStream.__init__()only initializedself._processorwhennoise_cancellationwas passed directly as aFrameProcessor.When
noise_cancellationwas a selector, the resolved value was only known later in_create_stream(), but that resolved processor was not stored on the input stream. This broke the path in_on_track_available()that injects stream info and credentials into the active processor.Changes
FrameProcessors as the active processor in_create_stream()_close_stream()FrameProcessorvalueaclose()for deterministic shutdown