Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import { concat } from '@langchain/core/utils/stream';
import { ChatOpenAI } from '@langchain/openai';

async function test() {
    const model = new ChatOpenAI({
        model: 'gpt-4o-audio-preview',
        modalities: ['text', 'audio'],
        audio: {
            format: 'pcm16',
            voice: 'alloy',
        },
    });

    const stream = await model.stream('Hello how are you');
    let result = undefined as AIMessageChunk | undefined;
    for await (const chunk of stream) {
        result = result ? concat(result, chunk) : chunk;
    }
    //@ts-expect-error test
    console.log(result?.additional_kwargs?.audio?.id)
}

test();

This logs audio_6871323ab2fc8191b2cf516af96cd851audio_6871323ab2fc8191b2cf516af96cd851 which then causes error:

400 Assistant audio audio_6871323ab2fc8191b2cf516af96cd851audio_6871323ab2fc8191b2cf516af96cd851 not found.

When asking the next message. I'm guessing the audio id should be audio_6871323ab2fc8191b2cf516af96cd851 instead

Error Message and Stack Trace (if applicable)

No response

Description

• i ask a text message through streaming
• i receive audio chunks
• i concat the chunks using the streaming utility
• i create a new AiMessage by passing the concat result in it
• the error happens

System Info

N/A