Customer Service Agent Evaluation

This directory contains the customer service agent and its evaluation tests.

Pinned to google-adk[eval]==2.3.0 (with uv.lock) for codelab reproducibility. Note for ADK 2.x: in the adk web UI the tab is now called Evals (was "Eval"), and evalsets created from the UI store invocation_events; hand-written eval.test.json files with tool_uses still work unchanged.

Running Evaluations

There are two ways to run evaluations:

1. Recommended: Programmatic Evaluation (Pytest)

This method is more robust and avoids naming conflicts with the ADK CLI.

From the project root directory (adk_eval/):

PYTHONPATH=. uv run pytest customer_service_agent/test_agent_eval.py

From this directory (customer_service_agent/):

PYTHONPATH=.. uv run pytest test_agent_eval.py

2. ADK CLI Evaluation

From the project root directory (adk_eval/):

uv run adk eval customer_service_agent customer_service_agent/eval.test.json --config_file_path customer_service_agent/test_config.json

From this directory (customer_service_agent/): Note: This may encounter naming conflicts if the file is named agent.py. It is recommended to use the Pytest method above.

uv run adk eval . eval.test.json --config_file_path test_config.json

3. Golden Dataset Evaluation

You can also run evaluation using the golden dataset generated by the ADK web UI.

CLI Method:

uv run adk eval customer_service_agent customer_service_agent/evalset780045.evalset.json --config_file_path customer_service_agent/test_config.json

Pytest Method:

PYTHONPATH=. uv run pytest customer_service_agent/test_golden_eval.py