Batch: re-added rate-limited objects stay in results.objs.all_responses as errors
Maintainers usually reply within 1 day
@feiiiiii5 is already working on this.
Since Nov 16, 2026.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start in weaviate/collections/batch/base.py at _BatchBase.__send_batch and reproduce with mock_tests/test_probe_readd_dep020.py using the mock gRPC fixtures. Verify the retry removes the stale ErrorObject, emits no Dep020 warning, and leaves four result entries without reading deprecated all_responses.
Written by the indexing model from the issue text.
Description
A rate-limited batch emits Dep020 in code that never reads all_responses, because the re-add in _BatchBase.__send_batch reads the deprecated property (weaviate/collections/batch/base.py:720). That line also filters by list position, while errors and uuids filter by the batch-wide index of the object. From the second chunk on the two differ, and the ErrorObject of the re-added object stays in _all_responses after its retry succeeds.
Mock gRPC server at 38f30c1 with the mock_tests/conftest.py fixtures, 4 objects through fixed_size(batch_size=2, concurrent_requests=1), the first object of the second chunk rate-limited once. The test never reads all_responses:
$ pytest mock_tests/test_probe_readd_dep020.py -q -s
Dep020 during batch=1 errors=0 len(_all_responses)=5 stale=1
mock_tests/test_probe_readd_dep020.py
import warnings
import weaviate
from weaviate.collections.classes.batch import ErrorObject
from weaviate.proto.v1 import batch_pb2, weaviate_pb2_grpc
from .conftest import MOCK_IP, MOCK_PORT, MOCK_PORT_GRPC, mock_class
class Svc(weaviate_pb2_grpc.WeaviateServicer):
calls = 0
def BatchObjects(self, request, context):
self.calls += 1
if self.calls == 2: # second chunk, once: rate-limit its first object
return batch_pb2.BatchObjectsReply(errors=[{"index": 0, "error": "failed with status: 503 error"}])
return batch_pb2.BatchObjectsReply()
def test_readd(weaviate_mock, start_grpc_server):
weaviate_mock.expect_request(f"/v1/schema/{mock_class['class']}").respond_with_json(mock_class)
weaviate_pb2_grpc.add_WeaviateServicer_to_server(Svc(), start_grpc_server)
client = weaviate.connect_to_local(port=MOCK_PORT, host=MOCK_IP, grpc_port=MOCK_PORT_GRPC)
try:
col = client.collections.use(mock_class["class"])
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
with col.batch.fixed_size(batch_size=2, concurrent_requests=1) as b:
for i in range(4):
b.add_object({"name": f"o{i}"})
r = col.batch.results.objs
dep = [x for x in w if "all_responses" in str(x.message)]
stale = [x for x in r._all_responses if isinstance(x, ErrorObject)]
print(f"\nDep020 during batch={len(dep)} errors={len(r.errors)} "
f"len(_all_responses)={len(r._all_responses)} stale={len(stale)}")
finally:
client.close()
Expected: no warning and 4 entries. Reading _all_responses and filtering on err.object_.index fixes both. #2166 already edits this function.
@tsmith023, you added Dep020. Would you rather take that one-line fix, or remove all_responses now that its Q4 2024 date has passed?
- Dominant language
- Python
- Stars
- 230
- Forks
- 173
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 23
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from weaviate/weaviate-python-client
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
weaviate/weaviate-python-client#2265 ·
Maintainers usually reply within 1 day
-
collections/batch: one object that fails client-side encoding fails every object sent with itPossibly taken @shadowmodder claimed this today. Openbug
Difficulty 3/5 1-2 days Newbie friendliness 72/100
weaviate/weaviate-python-client#2266 ·
Maintainers usually reply within 1 day
-
batch insert fails the whole batch with "ValueError: Unexpected type" when a value is a DecimalPossibly taken @steven-asearcy claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
weaviate/weaviate-python-client#2263 ·
Maintainers usually reply within 1 day
-
collections/batch/sync.py: after a fast hang-up reconnect the old stream's sender steals a request, hanging add_objectPossibly taken @shadowmodder claimed this 2 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
weaviate/weaviate-python-client#2261 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
weaviate/weaviate-python-client#2259 ·
Maintainers usually reply within 1 day
All issues in weaviate/weaviate-python-client
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
docling-project/docling-pipelines#669 ·
Maintainers usually reply within 1 day
-
blocked kind/enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
opendatahub-io/agent-eval-harness#329 · 2 comments · 1 reaction ·
-
cc_wrapper crashes with ValueError on Windows when a path is on another drivePossibly taken @TrapsterDK claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
tweag/rules_haskell#2492 ·
Maintainers usually reply within 1 day