BlobSourceInfo has no public setBlobExactSize() setter — causing high-volume WARN log on every queued ingest
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 58/100
Research direction
Start by reading com.microsoft.azure.kusto.ingest.source.BlobSourceInfo and the warning path in QueuedIngestClientImpl, then trace how URL-based ingest constructs and consumes the source. Clarify whether the intended resolution is a public size-setting path or a lower log level; done means URL ingestion no longer produces misleading high-volume WARN messages while successful ingestion behavior remains intact.
Written by the indexing model from the issue text.
Description
Describe the bug
Every call to ingestFromBlob() with a URL-based BlobSourceInfo emits a high-volume
WARN log:
Blob 'https://...' was sent for ingestion without specifying its raw data size
This fires once per blob ingestion event. In production this generates thousands
of warnings per minute, polluting logs and making it harder to identify real issues.
The root cause is that BlobSourceInfo has a private blobExactSize field with no
public setter. The only way to set it is via internal static factory methods
(fromFile, fromStream) which are not applicable for URL-based blob ingestion.
There is currently no public API path to set blobExactSize when constructing
BlobSourceInfo from a URL — yet the SDK logs a WARN on every call where it is unset.
To Reproduce
- Construct a
BlobSourceInfousing the URL constructor:BlobSourceInfo blobSourceInfo = new BlobSourceInfo("https://<storage-account>.blob.core.windows.net/<container>/<blob>"); - Call ingestFromBlob:
ingestClient.ingestFromBlob(blobSourceInfo, ingestionProperties); - Observe the following WARN log emitted on every call:
Blob 'https://...' was sent for ingestion without specifying its raw data size
Expected behavior
Either:
- Option A: Expose a public setBlobExactSize(long size) setter on BlobSourceInfo
so callers can provide the blob size when constructing via URL. We can obtain the size
via Azure Storage SDK (BlobProperties.getBlobSize()) and would happily pass it if a
setter were available. - Option B: Downgrade this log from WARN to DEBUG. A WARN implies something
is wrong with the call; in this case the blob is ingested successfully regardless —
the size is an optional optimization hint, not a correctness requirement. High-volume
WARN logs that do not indicate a real problem make production monitoring harder.
Screenshots
N/A
Setup (please complete the following information):
- JRE Version: OpenJDK 17
- SDK Version: kusto-ingest 7.0.6
Desktop (please complete the following information):
- OS: Linux (production), macOS (development)
- Version: Ubuntu 20.04 / macOS 14
Additional context
- The warning originates in QueuedIngestClientImpl and fires unconditionally when
blobExactSize is null on the BlobSourceInfo object. - Switching to fromFile() or fromStream() is not viable — our blobs already reside
in Azure Storage and downloading them locally before re-ingesting would add double
network cost, local disk pressure, and significant latency, defeating the purpose of
URL-based queued ingestion. - We checked BlobSourceInfo in SDK version 8.0.1 — setBlobExactSize() is still
not publicly exposed, so upgrading does not resolve this. - Related class: com.microsoft.azure.kusto.ingest.source.BlobSourceInfo
- Dominant language
- Java
- Stars
- 46
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Azure/azure-kusto-java
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Azure/azure-kusto-java#401 ·
-
Azure/azure-kusto-java#369 · 1 comment · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 32/100
Azure/azure-kusto-java#334 · 13 comments · 2 reactions ·
-
Azure/azure-kusto-java#232 · 3 comments · 1 assignee ·
-
enhancement ingest
Azure/azure-kusto-java#69 · 2 comments · 1 assignee ·
All issues in Azure/azure-kusto-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100