Skip to content

Conversation

omansour
Copy link

@omansour omansour commented Sep 4, 2025

summary

Empty tool in agent configuration let build tools to be executed.

Agent Configuration Used

{
  "$schema": "https://raw.githubusercontent.com/aws/amazon-q-developer-cli/refs/heads/main/schemas/agent-v1.json",
  "name": "test_execute_bash_allowed",
  "tools": [],
  "allowedTools": []
}

Test Results Comparison

Test 1: /tools Command

System q (1.15.0 Version) ❌

Tool       Permission
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Built-in:

Issue: Shows confusing empty "Built-in:" section with no helpful guidance.

Fixed Version ✅

Tool       Permission
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

No tools are currently enabled.

Refer to the documentation for how to add tools to your agent: https://github.com/aws/amazon-q-developer-cli/blob/main/docs/agent-format.md#tools-field

Fixed: Shows helpful message with documentation link.

Test 2: execute_bash Command

Command: "using execute_bash, run ls | head -n 1"

System q (Unfixed Version) ❌

🛠️  Using tool: execute_bash (trusted)
 ⋮ 
 ● I will run the following shell command: ls | head -n 1

book.toml

 ⋮ 
 ● Completed in 0.125s

> The command returned book.toml as the first item when listing the current directory contents.

Issue: Tool executes successfully despite tools: [] configuration - SECURITY VULNERABILITY (?)

Fixed Version ✅

Tool validation failed: 
No tool with "execute_bash" is found

> I don't have access to an execute_bash tool in my current environment. The available tools don't include bash command execution capabilities.

If you need to run ls | head -n 1 to see the first item in your current directory, you can run it directly in your terminal.

Fixed: Tool is properly blocked and user gets helpful guidance.

Files Modified

  1. crates/chat-cli/src/cli/chat/tool_manager.rs: Added validation in get_tool_from_tool_use() to check if tool is allowed by agent configuration
  2. crates/chat-cli/src/cli/chat/cli/tools.rs: Modified display logic to skip empty origins and show helpful message when no tools are configured

Both fixes maintain backward compatibility and only affect the specific case where an agent has tools: [] configured.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@omansour omansour marked this pull request as ready for review September 4, 2025 16:18
@@ -794,6 +794,17 @@ impl ToolManager {
status: ToolResultStatus::Error,
};

// Check if the tool is actually allowed by the agent configuration
if !self.schema.contains_key(&value.name) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's extra logic here for checking MCP tools just below so I don't think this would work as expected

Copy link
Author

@omansour omansour Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix is preventing the use of a build-in tools not activated in the config. However the code is a bit redundant maybe with the match you mention just below. It is possible to refactor a bit.

I am trying to deal with build-in tools and check theses tools within the config. Without that, the build-in tool will be available even if it is not listed in the tool entry of the config. Maybe it's wanted at product level ? cc @mschrage

here some other tests with the same config file

current q version

❯ echo -e "using execute_bash run touch toto.txt\n/tools\n/quit" |  q chat --agent test_execute_bash_allowed
🤖 You are chatting with claude-sonnet-4





🛠️  Using tool: execute_bash
 ⋮
 ● I will run the following shell command: touch toto.txt


Allow this action? Use 't' to trust (always allow) this tool for the session. [y/n/t]:



Tool       Permission
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔Built-in:



Allow this action? Use 't' to trust (always allow) this tool for the session. [y/n/t]:

=> the model try to execute the tool based on the default configuration. The tool is not listed in the tool entry of the json configuration. Seems to me like a bug

this PR version

❯ echo -e "using execute_bash run touch toto.txt\n/tools\n/quit" | /Users/omsr/Documents/myWorkspace/amazon-q-developer-cli/target/release/chat_cli chat --agent test_execute_bash_allowed
🤖 You are chatting with claude-sonnet-4





Tool validation failed:
No tool with "execute_bash" is found
> I don't have access to an execute_bash tool in this environment. The available tools don't include bash command execution capabilities.

You can run the command directly in your terminal:

bash
touch toto.txt


This will create an empty file named toto.txt in your current directory (/private/tmp).



Tool       Permission
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
No tools are currently enabled.

Refer to the documentation for how to add tools to your agent: https://github.com/aws/amazon-q-developer-cli/blob/main/docs/agent-format.md#tools-field

=> the model refuse to run the tool (and list that no tools are configured)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants