Multimodal writes fail entirely when embeddings are enabled (non-text piece crashes the add)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start at MemoryInterface.add_message_pieces_to_memory and trace its calls to generate_embedding_memory_data. Verify the behavior with a multimodal message containing text and an image_path piece: text should receive embeddings, while non-text pieces should still persist without an embedding row and the overall add should succeed.
Written by the indexing model from the issue text.
Description
what happens
MemoryInterface.add_message_pieces_to_memory with embeddings enabled iterates ALL received message pieces (minus not_in_memory) and calls generate_embedding_memory_data on each. that helper raises for non-text pieces, so a multimodal message (text plus an image_path piece) fails its whole write when embeddings are on, instead of persisting the message and embedding just the text.
repro
send any multimodal message with memory_embedding configured: the image_path piece hits the embedding generator, which rejects non-text data, and the entire add fails.
expected
text pieces embed; non-text pieces persist without an embedding row. embeddings only power text similarity search, so nothing is lost.
env: main 5503ecb
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 896
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 208
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/PyRIT
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
Similar issues
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
vllm-project/vllm-metal#822 ·
Maintainers usually reply within 1 day
-
vector-store
Difficulty 1/5 1-3 hours Newbie friendliness 90/100
mem0ai/mem0#7461 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug]: chunk_span_bounds and _validated_chunk_spans reject Pydantic models ChunkSpan and AudioFileOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
BasedHardware/omi#19047 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day