Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Late interaction normalization produces nonfinite float16 embeddings

Open
  • Is something wrong?
#825 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@IlyaKhalafi is already working on this.

Since Oct 15, 2026.

  • #826 by @IlyaKhalafi — open

Assessment

Difficulty
3/5
Estimated time
Half a day
Newbie friendliness
70/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
numpy, python
Domain
machine-learning

Research direction

Start with _post_process_onnx_output in the Colbert and LateOn late-interaction postprocessors, which compute L2 norms in float16. Run the reproduction from the issue, which builds an OnnxOutputContext directly, and check that the output has infinities or NaNs. Read the shared normalization helper that the issue says already handles float16 safely, and reuse its approach in both postprocessors. Done when float16 output keeps its dtype, the norms come out near [1, 0, 1], and existing padding, punctuation and query tests still pass.

Written by the indexing model from the issue text.

Description

Colbert._post_process_onnx_output and LateOn._post_process_onnx_output compute L2 norms in float16 when an ONNX model returns float16 embeddings. Small finite embeddings underflow to a zero norm, producing infinities. Zero vectors produce NaNs because the 1e-12 denominator floor also becomes zero in float16. Larger finite values can overflow the norm and become zero vectors.

I reproduced this on main at de502ca8 with NumPy 2.5.3 on Linux. The shared normalization helper already handles float16 safely, but these two postprocessors use their own normalization.

This isolates postprocessing without loading model weights.

import numpy as np
from fastembed.common.onnx_model import OnnxOutputContext
from fastembed.late_interaction.colbert import Colbert

model = object.__new__(Colbert)
model._ensure_tokenizer = lambda: None
model.skip_list = set()
model.pad_token_id = 0

vectors = np.full((1, 3, 128), 1e-5, dtype=np.float16)
vectors[0, 1] = 0
vectors[0, 2] = 100
context = OnnxOutputContext(
    model_output=vectors,
    attention_mask=np.ones((1, 3), dtype=np.int64),
    input_ids=np.array([[1, 2, 3]], dtype=np.int64),
)
result = next(iter(model._post_process_onnx_output(context)))
print(np.isfinite(result).all())
print(np.linalg.norm(result.astype(np.float32), axis=1))

The output contains infinities and NaNs, and the large vector has norm zero. Expected output is finite with norms near [1, 0, 1]. The same reproduction works with LateOn.

Float16 output should retain its dtype and be normalized using enough precision to avoid these errors. Existing padding, punctuation filtering and query behavior should be preserved.

Dominant language
Python
Stars
3.2k
Forks
276
Avg merge
13h 15m
Merged PRs (30d)
57

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from qdrant/fastembed

All issues in qdrant/fastembed

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.