Python: [Bug]: Duplicate messages in Handoff workflow

[cecheta]

Description

I ran the handoff_simple.py sample, but noticed that there were duplicate messages being sent within each agent's chat history.

Code Sample

I ran handoff_simple.py, however I used AzureOpenAIChatClient so I could clearly see the full list of messages sent to each agent.

Here is the full list of messages saved for the refund agent:

{"type": "message", "role": "user", "contents": [{"type": "text", "text": "Hello, I need assistance with my recent purchase.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "I'd be happy to help you with your recent purchase. Could you please tell me more about the issue you're experiencing? For example, are you looking to track your order, request a return, or inquire about a refund? This will help me connect you with the right specialist.", "additional_properties": {}}], "author_name": "triage_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "My order 1234 arrived damaged and the packaging was destroyed. I'd like to return it.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "I'm sorry your order arrived damaged. I've initiated a return for order 1234. You will receive detailed return instructions via email soon. \n\nIf you need further assistance with your refund or have additional questions, please let me know!", "additional_properties": {}}], "author_name": "return_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "Please also process a refund for order 1234.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "function_call", "call_id": "call_xBL1WNsjs2cshZzhgb4LBxIq", "name": "process_refund", "arguments": "{\"order_number\":\"1234\"}", "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "tool", "contents": [{"type": "function_result", "call_id": "call_xBL1WNsjs2cshZzhgb4LBxIq", "result": "Refund processed successfully for order 1234.", "items": [{"type": "text", "text": "Refund processed successfully for order 1234.", "additional_properties": {}}], "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "Your refund for order 1234 has been processed successfully. You should see the refunded amount on your original payment method soon.\n\nIf you have any more questions or need further assistance, feel free to ask!", "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "Hello, I need assistance with my recent purchase.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "I'd be happy to help you with your recent purchase. Could you please tell me more about the issue you're experiencing? For example, are you looking to track your order, request a return, or inquire about a refund? This will help me connect you with the right specialist.", "additional_properties": {}}], "author_name": "triage_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "My order 1234 arrived damaged and the packaging was destroyed. I'd like to return it.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "I'm sorry your order arrived damaged. I've initiated a return for order 1234. You will receive detailed return instructions via email soon. \n\nIf you need further assistance with your refund or have additional questions, please let me know!", "additional_properties": {}}], "author_name": "return_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "Please also process a refund for order 1234.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "function_call", "call_id": "call_xBL1WNsjs2cshZzhgb4LBxIq", "name": "process_refund", "arguments": "{\"order_number\":\"1234\"}", "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "tool", "contents": [{"type": "function_result", "call_id": "call_xBL1WNsjs2cshZzhgb4LBxIq", "result": "Refund processed successfully for order 1234.", "items": [{"type": "text", "text": "Refund processed successfully for order 1234.", "additional_properties": {}}], "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "Your refund for order 1234 has been processed successfully. You should see the refunded amount on your original payment method soon.\n\nIf you have any more questions or need further assistance, feel free to ask!", "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}
{"type": "message", "role": "user", "contents": [{"type": "text", "text": "Thanks for resolving this.", "additional_properties": {}}], "additional_properties": {}}
{"type": "message", "role": "assistant", "contents": [{"type": "text", "text": "You\u2019re very welcome! If you have any more questions or need help in the future, feel free to reach out. Have a great day!", "additional_properties": {}}], "author_name": "refund_agent", "additional_properties": {}}

It shows that messages are duplicated.

Package Versions

agent-framework-core: 1.0.0rc4

Python Version

3.13.11

Additional Context

No response

[LEDazzio01]

Root Cause Analysis

I traced through the message flow in HandoffAgentExecutor._run_agent_and_emit() and identified the source of the duplication.

The Problem

The base AgentExecutor._run_agent_and_emit() uses replace semantics for _full_conversation:

# Base AgentExecutor (replace)
self._full_conversation = [*self._cache, *(list(response.messages) if response else [])]

But HandoffAgentExecutor._run_agent_and_emit() uses extend semantics:

# HandoffAgentExecutor (extend) — line 453
self._full_conversation.extend(cleaned_incoming_messages)
# ...
# line 457-458
self._cache = list(self._full_conversation)
self._cache.extend(runtime_tool_messages)

This creates a duplication cycle:

1. After the first agent run, _cache is set to list(_full_conversation) (line 457), then cleared (line 483).
2. When handle_response is called, it adds the user message to _cache (line 577) and calls _run_agent_and_emit again.
3. On the second call, incoming_messages = list(self._cache) contains just the new user message — seems fine.
4. However, for agents that accumulate broadcast messages in _cache via the run() handler and are later re-activated through a handoff, _cache already contains messages that overlap with _full_conversation. The extend at line 453 blindly re-adds them.

The specific scenario: when an agent like refund_agent has accumulated all prior conversation via broadcasts in its _cache, then gets activated by handoff, _full_conversation.extend(cleaned_cache) works (empty → new). But if the agent is later re-activated (e.g., via a second handoff or re-entry), the accumulated broadcasts overlap with the existing _full_conversation, causing exact duplication.

Prior Art

This is the same class of bug that was fixed in PR #2867 ("Fix context duplication in handoff workflows when restoring from checkpoint"). The fundamental issue is using extend without deduplication-aware logic.

Proposed Fix

Only extend _full_conversation with genuinely new messages by comparing against what's already tracked:

# Instead of:
self._full_conversation.extend(cleaned_incoming_messages)

# Use:
cleaned_existing = clean_conversation_for_handoff(self._full_conversation)
new_cleaned = cleaned_incoming_messages[len(cleaned_existing):]
self._full_conversation.extend(new_cleaned)

This derives the count of already-tracked cleaned messages and only appends the remainder. It preserves the existing behavior for function call/result messages (which are kept in _full_conversation but stripped by clean_conversation_for_handoff).