Crash on incoming audio (0 Hz setting)

[denis-obukhov]

Describe the bug
Sometimes when I subscribe on an remote publication I get the crash:

           AURemoteIO.cpp:1091  failed: 1701737535 (enable 2, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>)
           AURemoteIO.cpp:1091  failed: 1701737535 (enable 2, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>)
           AVAEInternal.h:71    required condition is false: [AVAudioEngineGraph.mm:2161:_Connect: (IsFormatSampleRateAndChannelCountValid(format))]
*** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)'
*** First throw call stack:
(0x19131a5fc 0x18e8b1244 0x1914676e0 0x1abf7ec50 0x1ac0419a0 0x1ac045964 0x10706a490 0x1070671fc 0x107066bec 0x1071cb58c 0x1071472b0 0x106e6ee20 0x106e6f244 0x106e6c24c 0x106eeb958 0x106f7743c 0x106f76334 0x106f779a8 0x21bffe7d0 0x21bffe480)
libc++abi: terminating due to uncaught exception of type NSException

Both listener and streamer are my local iOS devices connected to Xcode. I stream opus audio only.

As far as I understand this is because of 0 Hz audio settings.

SDK Version
v2.1.0
LiveKit Cloud

iOS/macOS Version
iOS 18.3

Xcode Version
Version 16.2 (16B5100e)

Steps to Reproduce
It happens randomly. I've never seen it before. I used an older version of the SDK (2.0.8). Probably, moving from background <-> foreground affects this.
What I'm doing is basically this:

public func room(_ room: Room, participant: RemoteParticipant, didPublishTrack publication: RemoteTrackPublication) {
//... some logic
try await publication.set(subscribed: true)
}

Expected behavior
No crash

[hiroshihorie]

Thanks for the report, will investigate this one today.
Please let me know if you notice any other clues to reproduce.

[hiroshihorie]

Hmm I can't reproduce it, but I have seen this when the AudioSession config was not correct.
Do you do any custom audio session category modification ?

[denis-obukhov]

@hiroshihorie Yeah. I have this modified config initially copied from v2.0.8. Now, I see that it has been changed inside the SDK.

      AudioManager.shared.customConfigureAudioSessionFunc = { newState, oldState in
            DispatchQueue.liveKitWebRTC.async { [weak self] in
                guard let self else { return }
                
                // prepare config
                let configuration = LKRTCAudioSessionConfiguration.webRTC()
                configuration.category = AVAudioSession.Category.playAndRecord.rawValue
                configuration.mode = AVAudioSession.Mode.voiceChat.rawValue
                configuration.categoryOptions = [
                    .allowBluetooth,
                    .duckOthers,
                    .defaultToSpeaker
                ]
                
                var setActive: Bool?
                
                if newState.trackState != .none, oldState.trackState == .none {
                    // activate audio session when there is any local/remote audio track
                    setActive = true
                } else if newState.trackState == .none, oldState.trackState != .none {
                    // deactivate audio session when there are no more local/remote audio tracks
                    setActive = false
                }
                
                // configure session
                let session = LKRTCAudioSession.sharedInstance()
                session.lockForConfiguration()
                // always unlock
                defer { session.unlockForConfiguration() }
                
                do {
                    if let setActive {
                        try session.setConfiguration(configuration, active: setActive)
                    } else {
                        try session.setConfiguration(configuration)
                    }
                } catch {
                    self.print("Failed to configure audio session with error: \(error)")
                }
            }
        }

[hiroshihorie]

Ok that won't work, it must be sync not async.
Do you need the custom func anyways ? if not I recommend to not set custom func and let the SDK handle it automatically.

The new audio engine will now precisely invoke the method right before it starts,
so by the time the func returns it must be already configured.

It's too late if it's async and the engine will fail to start.

[denis-obukhov]

@hiroshihorie Got it! Thanks. But I do need to always have playAndRecord category in order to work correctly with Apple Push To Talk framework I use along with LiveKit. Moreover, Apple says that I don't need to activate the audio session on my own. The framework does it automatically by calling func channelManager(_ channelManager:, didActivate audioSession:)`

[hiroshihorie]

If you simply want playAndRecord all the time:

1. Disable SDKs automatic configuration by calling : AudioManager.shared.set(engineObservers:[]) . (custom config func will not be invoked also)
2. Simply configure your audio session at app startup :

    let session = AVAudioSession.sharedInstance()
    try session.setCategory(.playAndRecord, mode: .voiceChat)
    try session.setActive(true)

Would that work for you ?

[denis-obukhov]

Not really. Meanwhile, it works fine for me on v2.0.19 using the customConfigureAudioSessionFunc approach.

Moreover, having AudioManager.shared.set(engineObservers: []) and activating the audio session on the launch I've just encountered this crash again, but this time on an iPad Air (5th Gen) running iOS 18.2.1, without switching to the background state—just a freshly launched app with Xcode connected. It reproduces every time, right after subscribing on an incoming publication.

The crash:

           AURemoteIO.cpp:1091  failed: 1701737535 (enable 2, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>)
           AURemoteIO.cpp:1091  failed: 1701737535 (enable 2, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>)
           AVAEInternal.h:71    required condition is false: [AVAudioEngineGraph.mm:2161:_Connect: (IsFormatSampleRateAndChannelCountValid(format))]

[kalavski]

I checked also the example app on macCatalyst with LiveKit version 2.2.1 and there bug also exist there.
Tapping mic button and make it enabled causes error.

*** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000018878ae80 __exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x0000000188272cd8 objc_exception_throw + 88
	2   CoreFoundation                      0x000000018878ad70 +[NSException exceptionWithName:reason:userInfo:] + 0
	3   AVFAudio                            0x00000001e5b46db0 _ZN18AVAudioEngineGraph8_ConnectEP19AVAudioNodeImplBaseS1_jjP13AVAudioFormat + 348
	4   AVFAudio                            0x00000001e5b70dd8 _ZN17AVAudioEngineImpl7ConnectEP11AVAudioNodeS1_mmP13AVAudioFormat + 1620
	5   AVFAudio                            0x00000001e5b74fc4 -[AVAudioEngine connect:to:format:] + 128
	6   LiveKitWebRTC                       0x00000001020ca25c _ZNK7cricket4Port13StunDscpValueEv + 44748
	7   LiveKitWebRTC                       0x00000001020c67d4 _ZNK7cricket4Port13StunDscpValueEv + 29764
	8   LiveKitWebRTC                       0x00000001020c6ad4 _ZNK7cricket4Port13StunDscpValueEv + 30532
	9   LiveKitWebRTC                       0x00000001021a752c _ZN6webrtc41CreateVideoEncoderSoftwareFallbackWrapperENSt4__Cr10unique_ptrINS_12VideoEncoderENS0_14default_deleteIS2_EEEES5_b + 183872
	10  LiveKitWebRTC                       0x0000000101ecc570 _ZN6webrtc10NV12Buffer12MutableDataYEv + 45244
	11  LiveKitWebRTC                       0x0000000101fd479c _ZN3rtc6Thread8DispatchEN4absl12AnyInvocableIFvvOEEE + 160
	12  LiveKitWebRTC                       0x0000000101fd36a0 _ZN3rtc6Thread15ProcessMessagesEi + 220
	13  LiveKitWebRTC                       0x0000000101fd4d08 _ZN3rtc6Thread6PreRunEPv + 96
	14  libsystem_pthread.dylib             0x000000010090e9ac _pthread_start + 136
	15  libsystem_pthread.dylib             0x0000000100918cfc thread_start + 8\n
)
An uncaught exception was raised
required condition is false: IsFormatSampleRateAndChannelCountValid(format)
(
	0   CoreFoundation                      0x000000018878ae80 __exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x0000000188272cd8 objc_exception_throw + 88
	2   CoreFoundation                      0x000000018878ad70 +[NSException exceptionWithName:reason:userInfo:] + 0
	3   AVFAudio                            0x00000001e5b46db0 _ZN18AVAudioEngineGraph8_ConnectEP19AVAudioNodeImplBaseS1_jjP13AVAudioFormat + 348
	4   AVFAudio                            0x00000001e5b70dd8 _ZN17AVAudioEngineImpl7ConnectEP11AVAudioNodeS1_mmP13AVAudioFormat + 1620
	5   AVFAudio                            0x00000001e5b74fc4 -[AVAudioEngine connect:to:format:] + 128
	6   LiveKitWebRTC                       0x00000001020ca25c _ZNK7cricket4Port13StunDscpValueEv + 44748
	7   LiveKitWebRTC                       0x00000001020c67d4 _ZNK7cricket4Port13StunDscpValueEv + 29764
	8   LiveKitWebRTC                       0x00000001020c6ad4 _ZNK7cricket4Port13StunDscpValueEv + 30532
	9   LiveKitWebRTC                       0x00000001021a752c _ZN6webrtc41CreateVideoEncoderSoftwareFallbackWrapperENSt4__Cr10unique_ptrINS_12VideoEncoderENS0_14default_deleteIS2_EEEES5_b + 183872
	10  LiveKitWebRTC                       0x0000000101ecc570 _ZN6webrtc10NV12Buffer12MutableDataYEv + 45244
	11  LiveKitWebRTC                       0x0000000101fd479c _ZN3rtc6Thread8DispatchEN4absl12AnyInvocableIFvvOEEE + 160
	12  LiveKitWebRTC                       0x0000000101fd36a0 _ZN3rtc6Thread15ProcessMessagesEi + 220
	13  LiveKitWebRTC                       0x0000000101fd4d08 _ZN3rtc6Thread6PreRunEPv + 96
	14  libsystem_pthread.dylib             0x000000010090e9ac _pthread_start + 136
	15  libsystem_pthread.dylib             0x0000000100918cfc thread_start + 8
)
FAULT: com.apple.coreaudio.avfaudio: required condition is false: IsFormatSampleRateAndChannelCountValid(format); (user info absent)

[hiroshihorie]

@kalavski Does this happen when simply enabling the mic ?

[kalavski]

I used LiveKit example app from your repository and from what I see it is just enabling the mic, nothing more.

let options = AudioCaptureOptions(noiseSuppression: false, highpassFilter: false)
try await room.localParticipant.setMicrophone(enabled: !isMicrophoneEnabled, captureOptions: options)

[hiroshihorie]

@kalavski Thanks... that's odd I can't reproduce it.
Anything unique to your environment, what input device is used ? Any devices connect yo your Mac ?
Can you reproduce this 100% of the time ?

Thank you for your information, this helps debugging this issue.

[kalavski]

@hiroshihorie Thank you for your help.

There is one thing that I changed in example app environment and it is a multipath capabilities. I removed that capability from the project.

Yes, it happens every time I tap mic button.

Mac version: MacBook Pro with M1 Pro
macOS: Sequoia 15.3.1

There is nothing connected to my Mac.

[hiroshihorie]

@kalavski The Multipath Entitlement ? I don't think it's related.

Do you remember if the mic permission dialog appeared ? Is this granted ?