Clarification regarding batch support

[cognivore]

Why?

Many individuals use Batch API of OpenAI to dramatically reduce costs of operation trading off the soft real time nature of responses.

What?

• Clarify the state of batch support in this library (cursory look returns nothing)
• If it's not supported, clarify what would it take for an external contributor to support it and get it merged

[jeremychone]

Thank you for the question—this is an important feature that should be added.

• It wasn’t included initially because I wanted to ensure request/response handling was solid first.

• I need to review it further. I usually study how different providers handle batch support, then design a way to standardize it. It may be more complex than a simple request/response model, but it should be doable.

I will follow up on this ticket.

[ram-from-tvl]

Hi @jeremychone
IF the issue is open to contributors. I would like to work on this.
Could you please assign this to me.
Thank you!