Python: Bug: ChatMessageContent.from_element drops AudioContent and BinaryContent into raw TextContent
Maintainers usually reply within 3 days
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
Research direction
Start in python/semantic_kernel/contents/chat_message_content.py by reading TAG_CONTENT_MAP and the from_element() handling for content tags. Check how AudioContent and BinaryContent serialize and deserialize, then verify XML and prompt round trips preserve their item types. Also check the semantic_kernel.contents.__all__ export for BinaryContent; done means both types round-trip without becoming TextContent.
Written by the indexing model from the issue text.
Description
Describe the bug
When ChatMessageContent containing AudioContent or BinaryContent is converted to an XML Element and deserialized via ChatMessageContent.from_element(), or round-tripped through prompt rendering (ChatHistory.to_prompt() / ChatHistory.from_rendered_prompt()), the AudioContent and BinaryContent items are corrupted into TextContent containing literal XML tags (e.g. <audio>...</audio>), and a warning is logged:
Unknown tag "audio" in ChatMessageContent, treating as text
Where
In python/semantic_kernel/contents/chat_message_content.py:
TAG_CONTENT_MAP includes tags for annotation, text, file_reference, function_call, function_result, image, reasoning, and streaming variants, but omits AUDIO_CONTENT_TAG ("audio") and BINARY_CONTENT_TAG ("binary").
Also, BinaryContent is omitted from semantic_kernel.contents.__all__.
To Reproduce
from semantic_kernel.contents import AudioContent, ChatMessageContent, AuthorRole
ac = AudioContent(data=b"test_audio", mime_type="audio/mpeg")
cmc = ChatMessageContent(role=AuthorRole.USER, items=[ac])
elem = cmc.to_element()
restored = ChatMessageContent.from_element(elem)
assert cmc == restored
# Fails:
# WARNING:root:Unknown tag "audio" in ChatMessageContent, treating as text
# AssertionError: ChatMessageContent(..., items=[AudioContent(...)]) != ChatMessageContent(..., items=[TextContent(text='<audio>data:audio/mpeg;,test_audio</audio>')])
Expected behavior
ChatMessageContent.from_element() correctly deserializes <audio> into AudioContent and <binary> into BinaryContent, preserving item types across prompt and XML round trips.
Platform
- Language: Python
- Source: main branch
- OS: Mac / Linux / Windows
- Dominant language
- C#
- Stars
- 28.8k
- Forks
- 4.9k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 48
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/semantic-kernel
-
Python: Bug: Redis search returns None for every field that declares a storage_name, because RETURN lists the declared namePossibly taken @wasim-builds claimed this 9 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
microsoft/semantic-kernel#14753 · 1 comment ·
Maintainers usually reply within 3 days
-
.Net: gpt-image-1 is the default image model in the .NET OpenAI connector, and OpenAI shuts it down on October 23Possibly taken @nightcityblade claimed this 46 days ago. Open.NET triage
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/semantic-kernel#14526 · 1 comment ·
Maintainers usually reply within 3 days
-
Python: VolatileMemoryStore.get_batch and get_nearest_matches ignore with_embeddings=False (deepcopy result is discarded)Possibly taken @VANDRANKI claimed this 48 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
microsoft/semantic-kernel#14522 ·
Maintainers usually reply within 3 days
-
.Net: Bug: BinaryContent does not decode the %xx escapes of a non-base64 data URIPossibly taken @Laurianti claimed this 48 days ago. Open.NET triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
microsoft/semantic-kernel#14518 ·
Maintainers usually reply within 3 days
-
Python: FunctionCallContent.combine_arguments drops a streamed "{}" chunk, producing invalid JSON argumentsPossibly taken @VANDRANKI claimed this 49 days ago. Openpython triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
microsoft/semantic-kernel#14512 · 2 comments ·
Maintainers usually reply within 3 days
All issues in microsoft/semantic-kernel
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Support Flux ignore rules (drift detection ignore paths) in Microsoft.KubernetesConfiguration schemaOpen
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Azure/azure-resource-manager-schemas#3921 ·
Maintainers usually reply within 1 day
-
text-align: justify is ignored in HTML to PDF conversionPossibly taken @PrzemyslawKlys claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
asyncapi/modelina#2709 · 1 comment ·
Maintainers usually reply within 1 day