Streaming commit transaction does embedding and fuzzy index work that could be pipelined
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Start in conversation_base.py at _commit_batch_streaming (line 434), then trace _update_secondary_indexes_incremental through _update_message_index_incremental and _update_related_terms_incremental. Separate embedding work from the transaction and defer fuzzy index updates until all batches complete; done means commit performs only database writes while ingested data remains correct and query-time fuzzy indexing is available afterward.
Written by the indexing model from the issue text.
Description
_commit_batch_streaming (conversation_base.py line 434) opens a transaction and calls _update_secondary_indexes_incremental inside it. That function does two expensive things:
-
Embedding generation —
_update_message_index_incremental→message_index.add_messages()→text_location_index.add_text_locations()→_embedding_index.add_texts(). These are API calls to the embedding model, happening inside the DB transaction. -
Fuzzy index term embeddings —
_update_related_terms_incrementalcollects new terms from semantic refs and callsfuzzy_index.add_terms(), which generates an embedding per unique term. Many terms repeat across batches; theCachingEmbeddingModelhelps but the per-batch overhead of collecting and checking is still there.
The two-stage pipeline already overlaps LLM extraction(N+1) with commit(N). But the commit phase itself is slower than necessary because of these embedding calls. Pre-computing embeddings alongside knowledge extraction (before the transaction opens) would keep the commit phase to pure DB writes. The fuzzy index could also be deferred to a single pass after all batches complete, since it's only needed for query-time — not for correctness of the ingested data.
- Dominant language
- Python
- Stars
- 884
- Forks
- 75
- Avg merge
- 3h 2m
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/typeagent-py
-
Difficulty 3/5 1-2 days Newbie friendliness 78/100
microsoft/typeagent-py#319 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 52/100
microsoft/typeagent-py#308 · 6 comments ·
-
microsoft/typeagent-py#307 · 2 comments · 1 assignee ·
-
microsoft/typeagent-py#298 · 7 comments · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
microsoft/typeagent-py#171 · 1 comment ·
All issues in microsoft/typeagent-py
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100