Python: [Bug]: Structured output JSON duplication / invalid JSON with GroupChat manager using AzureAIAgentClient (follow-up to #3049)

[legendgokul]

Description

Hi maintainers 👋

This issue is a follow-up to #3049, which was closed but the underlying problem is still reproducible for me even after validating with the latest package version.

Original issue:
#3049

Environment :
Runtime: Python 3.12.10 ,
Client used: AzureAIAgentClient (not AzureOpenAIChatClient)
Model: gpt-4.1
Workflow: GroupChatBuilder with manager agent
Package version: agent-framework ( 1.0.0b260107 )

Current Setup :
In the current setup we have created multiple agent with mcp tools attached to it, these agent i have created using agent-framework package, hence i able to see the tools added under action section, compared to when i create mcp using VScode extension where MCP get added to knowledge section of the agent.

Code Sample

# NOTE:
		# - All identifiers, tokens, URLs, and tool definitions are redacted
		# - Issue reproduced with latest package version
		# - Manager agent uses strict json_schema with streaming enabled
		
		#Code used which pulls AIFoundry agent :
		mcp_tools = MCPStreamableHTTPTool(
                name=mcp_tool_name,
                url=mcp_tool_url,
                approval_mode="never_require",
                allowed_tools = [<array of tools>], #redacted
                headers={"Authorization": "Bearer <redacted>"},
            )
        Agent1 = ChatAgent(            
                chat_client= AzureAIAgentClient(
                    credential=credential,
                    project_endpoint = os.getenv("AZURE_PROJECT_ENDPOINT"),
                    model_deployment_name = os.getenv("AZURE_MODEL_DEPLOYMENT"),         
                    agent_id="<agent_Id>"  #redacted
                ),    
                name="Agent1", #redacted
                tools=[mcp_tools],                       
            )
		
		
		#Code used to create and run the orchestration:
		workflow = (
                GroupChatBuilder()
                .set_manager(cordinatorV2, display_name="Orchestrator")
                .participants([Agent1,<AgentList removed since they are sensitive])  
                .build()
            )

        result = []
        final_conversation: list[ChatMessage] = []

        user_message = ChatMessage(
                    role=Role.USER,
                    text=context
                )
        
        currentMessage = ""
        max_retries = 3
        retry_count = 0
        while retry_count < max_retries:
            final_conversation: list[ChatMessage] = []
            last_executor_id: str | None = None
            try :
                async for event in workflow.run_stream(user_message):
                    if isinstance(event, AgentRunUpdateEvent):
                        currentAgent = event.data.author_name  # we are getting current executing agent name
                        if currentAgent != last_executor_id:  # this is to check if the event is from same agent.
                            # if new agent , create last agent response object
                            if last_executor_id is not None :
                                result.append({
                                "agent": last_executor_id,
                                "message": currentMessage
                            })
                                
                            
                            print(f"\n{currentAgent}:", end=" ", flush=True)
                            # reset variables for new agent ID
                            last_executor_id = currentAgent
                            currentMessage = ""
                        
						print(event.data, end="", flush=True)
                        currentMessage += event.data.text
                    elif isinstance(event, WorkflowOutputEvent):
                            result.append({
                                    "agent": last_executor_id,
                                    "message": currentMessage
                                })
                            final_conversation = cast(list[ChatMessage], event.data)
                            result.append({
                                    "agent": "Final",
                                    "message": event.data[-1].text
                                })
                break
            except Exception as e:
                retry_count += 1
                error_msg = str(e)
                
                if "Invalid JSON" in error_msg or "ManagerSelectionResponse" in error_msg:
                    print(f"\n\n JSON validation error (attempt {retry_count}/{max_retries})")
                    print(f"Error: {error_msg[:200]}")
                    
                    if retry_count < max_retries:
                        print(f"Retrying...\n")
                        await asyncio.sleep(1)  # Brief delay before retry
                        continue
                    else:
                        print(f"\n Failed after {max_retries} attempts.")
                        print("\n Recommendation: Update Coordinator agent instructions in AI Foundry:")
                        print(" Add: 'Return EXACTLY ONE JSON object. Never return multiple JSON objects.'")
                        raise
                else:
                    # Different error - don't retry
                    raise

Error Messages / Stack Traces

Error Info :
Error 1 : 

Coordinator: {"selected_participant":"Agent1","instruction":"<instruction>","finish":false,"final_message":null}executor broke JSON is invalid: Expecting value: line 1 column 1 (char 0)

Error 2 : Coordinator/ manager agent JSON is getting duplicated as shown below

Coordinator: {"selected_participant":"<agentname>","instruction":"<instruction>","finish":false,"final_message":null}{"selected_participant":"<agentname>","instruction":"<instruction>","finish":false,"final_message":null}executor broke 1 validation error for ManagerSelectionResponse
  Invalid JSON: trailing characters at line 1 column 208 [type=json_invalid, input_value='{"selected_participant":...e,"final_message":null}', input_type=str]
   For further information visit https://errors.pydantic.dev/2.12/v/json_invalid

Package Versions

1.0.0b260107

Python Version

3.12.10

Additional Context

No response

[moonbox3]

I've been seeing this type of duplication these days with the AzureAIClient / AzureAIAgentClient. I haven't been able to pinpoint it yet - however, it seems like we have some sort of retry enabled where we then concatenate the JSON results, and thus break downstream operations. @dmytrostruk do you have any thoughts around this, having worked with these clients quite a bit?