Skip to content

MethodToolCallback does not handle invalid arg gracefully #3924

@ls-rein-martha

Description

@ls-rein-martha

Bug description

Similar to: #2857 (but this is the MethodToolCallback impl)

When LLM intent to call a tool, there is possibility that the provided argument is invalid.
For example:

  • The tool argument is an enum
  • The LLM provide invalid enum

When this occur it should throw ToolExecutionException instead of IllegalArgumentException. Or it might even be a good idea to change ToolExecutionExceptionProcessor process to all Exception instead of ToolExecutionException so other possible edge cases can be handled gracefully easily

Stack trace:

java.lang.IllegalArgumentException: No enum constant __INVALID_ENUM_NAME__
	at java.base/java.lang.Enum.valueOf(Enum.java:293)
	at org.springframework.ai.util.json.JsonParser.toTypedObject(JsonParser.java:152)
	at org.springframework.ai.tool.method.MethodToolCallback.buildTypedArgument(MethodToolCallback.java:150)
	at org.springframework.ai.tool.method.MethodToolCallback.lambda$buildMethodArguments$1(MethodToolCallback.java:139)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)

Environment
Spring AI 1.0.0
Java 17
Azure OpenAI gpt-4o

Steps to reproduce
Create a tool that accepts an enum as a parameter, let's say:

  • ONE_DAY
  • TWO_DAY
  • THREE_DAY

If you ask LLM for FOUR_DAY or similar, it might try to call the tool, even if the input schema doesn't list it so

Expected behavior
It should throw ToolExecutionException so it can be sent back to model, or handle gracefully at ToolExecutionExceptionProcessor.

Workaround (create a custom MethodToolCallback with a delegate):


@RequiredArgsConstructor
public class CustomMethodToolCallback implements ToolCallback {

    @Delegate
    private final MethodToolCallback delegate;

    @Override
    public String call(String toolInput) {
        return call(toolInput, null);
    }

    @Override
    public String call(String toolInput, @Nullable ToolContext toolContext) {
        try {
            return delegate.call(toolInput, toolContext); // Not perfect, but acceptable
        } catch (IllegalArgumentException iae) {
            throw new ToolExecutionException(getToolDefinition(), iae);
        }
    }
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions