Skip to content

Conversation

suryabdev
Copy link
Contributor

@suryabdev suryabdev commented Oct 2, 2025

When the output token length is smaller than required for a particular use case, the user sees parsing errors. It is unclear to them that the issue might be with the shorter token length. They might see errors like

Error while parsing tool call from model output: The model output does not contain any JSON blob.

for ToolCallingAgents and

The code blob is invalid, because the regex pattern ```(?:py|python)?\n(.*?)\n``` was not found in code_blob

for CodeAgents

Issues: #1783, #1732, etc. There are older threads were the issue is mentioned. #201 (comment), Workaround was to increase the output tokens

That happens with some models. Sometimes it can be tweaked by increasing max_tokens or num_ctx

It is also mentioned that longer contexts are better in an example in the docs

num_ctx=8192, # ollama default is 2048 which will fail horribly. 8192 works for easy tasks, more is better. Check https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator to calculate how much VRAM this will need for the selected model.

This PR tries to

  1. Improve error logs
  2. Add a note in the documentation

return json_data, json_blob[:first_accolade_index]
except IndexError:
raise ValueError("The model output does not contain any JSON blob.")
raise ValueError("The model output does not contain any JSON blob. Try increasing the maximum output token length.")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the tool calling agent, I've added some details to the error message. This gets fed back to the ReAct loop but only the user can edit the model parameters. Maybe we should print it at the end of the trace?

For the CodeAgent, Shall I add a similar error in parse_code_blobs?

Your code snippet is invalid, because the regex pattern {code_block_tags[0]}(.*?){code_block_tags[1]} was not found in it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should print it at the end of the trace?

Actually thinking again, we shouldn't pass this part to the ReAct loop. It could confuse the model
Maybe print a warning message that doesn't get added to the agent's memory

@suryabdev
Copy link
Contributor Author

Perhaps in the "How to debug your agent" section, We could add another point
https://huggingface.co/docs/smolagents/en/tutorials/building_good_agents#how-to-debug-your-agent
An expanded/more clear version of the following 👇

5. Increase token length

If you experience issues with parsing code (for CodeAgents) or JSON objects (for ToolCallingAgents) like the following, it could be due to insufficient output token length. Please adjust your model's parameters accordingly.

Error while parsing tool call from model output: The model output does not contain any JSON blob.

for ToolCallingAgents and

The code blob is invalid, because the regex pattern ```(?:py|python)?\n(.*?)\n``` was not found in code_blob

for CodeAgents

@suryabdev
Copy link
Contributor Author

cc: @albertvillanova (For review)
I have some clarifications, Please take a look when you are free

@suryabdev suryabdev changed the title Improve error handling for short output token lengths Improve error messaging for short output token lengths Oct 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant