Skip to content

Conversation

suryabdev
Copy link
Contributor

Fix for #1572

Re-implemented the skipped test_change_tools_after_init tests

  • Changed TestToolCallingAgent/test_change_tools_after_init to use a ToolCallingAgent instead of a CodeAgent
  • Replace fake_tool with a tool of the same name

Testing

Ran the test multiple times using the pytest-repeat plugin

> pytest --count=50 tests/test_agents.py::TestCodeAgent::test_change_tools_after_init
=================================================================== test session starts ====================================================================
platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 -- /home/suryab/coda/venv/smolagents/bin/python3
cachedir: .pytest_cache
rootdir: /home/suryab/coda/smolagents
configfile: pyproject.toml
plugins: anyio-4.10.0, timeout-2.4.0, repeat-0.9.4, datadir-1.8.0
collected 50 items

tests/test_agents.py::TestCodeAgent::test_change_tools_after_init[1-50] PASSED
tests/test_agents.py::TestCodeAgent::test_change_tools_after_init[2-50] PASSED
0.01s call     tests/test_agents.py::TestCodeAgent::test_change_tools_after_init[34-50]

(100 durations < 0.005s hidden.  Use -vv to show these durations.)
==================================================================== 50 passed in 0.79s ====================================================================
> pytest --count=50 tests/test_agents.py::TestToolCallingAgent::test_change_tools_after_init
=================================================================== test session starts ====================================================================
platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 -- /home/suryab/coda/venv/smolagents/bin/python3
cachedir: .pytest_cache
rootdir: /home/suryab/coda/smolagents
configfile: pyproject.toml
plugins: anyio-4.10.0, timeout-2.4.0, repeat-0.9.4, datadir-1.8.0
collected 50 items

tests/test_agents.py::TestToolCallingAgent::test_change_tools_after_init[1-50] PASSED
tests/test_agents.py::TestToolCallingAgent::test_change_tools_after_init[2-50] PASSED
...
0.01s call     tests/test_agents.py::TestToolCallingAgent::test_change_tools_after_init[16-50]

(100 durations < 0.005s hidden.  Use -vv to show these durations.)
==================================================================== 50 passed in 0.52s ====================================================================

And ran all tests in the file

> pytest tests/test_agents.py
=================================================================== test session starts ====================================================================
platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 -- /home/suryab/coda/venv/smolagents/bin/python3
cachedir: .pytest_cache
rootdir: /home/suryab/coda/smolagents
configfile: pyproject.toml
plugins: anyio-4.10.0, timeout-2.4.0, repeat-0.9.4, datadir-1.8.0
collected 94 items

tests/test_agents.py::TestAgent::test_fake_toolcalling_agent PASSED
...
0.01s call     tests/test_agents.py::TestAgent::test_setup_agent_with_empty_toolbox

(211 durations < 0.005s hidden.  Use -vv to show these durations.)
============================================================== 94 passed, 1 warning in 12.26s ==============================================================


@tool
def fake_tool_2() -> str:
def fake_tool_1() -> str:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For

Test is not properly implemented (GH-1255) because fake_tools should have the same name.

Is this the expected change? Should I use the class method instead to create a fake tool with the same name?
If I understand correctly we want to replace an existing tool
https://huggingface.co/docs/smolagents/en/tutorials/tools#manage-your-agents-toolbox

reason="Test is not properly implemented (GH-1255) because fake_tools should have the same name. "
"Additionally, it uses CodeAgent instead of ToolCallingAgent (GH-1409)"
)
def test_change_tools_after_init(self):
Copy link
Contributor Author

@suryabdev suryabdev Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #1571, it was mentioned that the tests were flaky

even though they may pass or fail inconsistently.

Do you remember the error message in the failed runs? I ran the tests multiple times (Ran the entire tests/test_agents.py file a few times, mentioned details in the PR description). test_change_tools_after_init used to always pass

@suryabdev
Copy link
Contributor Author

cc: @albertvillanova (For review)
Added some questions, Please take a look when you are free

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant