Skip to content

Conversation

ricken07
Copy link
Contributor

Improving the content methods of the DefaultChatClient

This pull request contains a proposal for updating the content methods of the DefaultChatClient.

Improving the content methods of the DefaultChatClient

This pull request contains a proposal for updating the content methods of the DefaultChatClient class.

Here's a use case to avoid making another request on the content method:

I want to retrieve the metadata from the chat response and then use content to retrieve the final response.

In this use case, as shown in the example, I use the call method, which returns a CallResponseSpec. This CallResponseSpec allows me to retrieve the metadata by calling the chatResponse method (step 1). I then use the content method directly to obtain the final response via the spec. The current implementation makes a second request to the API to retrieve a ChatResponse that has already been obtained in step (1), which means there are two calls to the spec for a single specific action, which is excessive.

This is why I suggest checking in the spec whether a call to the model has already been made before launching a new one.

`
var responseSpec = chatclient.prompt()
.user("This is a test")
.call();

var metadata = responseSpec.chatResponse().getMetadata();

var response = responseSpec.content();
`
@tzolov this use case makes sense if we want to retrieve the metadata for billing or other purposes and use the content method to retrieve the final response directly and avoid consuming an additional request.

@csterwa csterwa added this to the 1.0.0-RC1 milestone Sep 25, 2024
@markpollack markpollack modified the milestones: 1.0.0-RC1-triage, 1.0.x May 8, 2025
@markpollack markpollack removed this from the 1.1.0.M1 milestone Sep 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants