Llama.embed() calls LlamaBatch.add_sequence with old 3-arg signature; missing logits_array
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 65/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- python
- Domain
- backend-api-design
Research direction
The issue is in llama_cpp/llama.py around line 1678, where Llama.embed() calls _batch.add_sequence with three arguments. Compare with the correct four-argument call in llama_cpp/llama_embedding.py around line 262. Update the call to match the signature, using the token array, position array, sequence IDs, and logits array. Test by running the provided reproduction script with a model that supports embeddings.
Written by the indexing model from the issue text.
Description
Prerequisites
- I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
Llama.embed() should successfully compute embeddings when called on a model constructed with embeddings=True.
Current Behavior
Llama.embed() raises a TypeError immediately, before any embedding is computed:
TypeError: LlamaBatch.add_sequence() missing 1 required positional argument: 'logits_array'
The cause: Llama.embed() in llama_cpp/llama.py (around line 1678) calls add_sequence with three positional arguments:
self._batch.add_sequence(tokens, p_batch, logits_all)
But LlamaBatch.add_sequence in llama_cpp/_internals.py (around line 1013) requires four:
def add_sequence(
self,
token_array: Sequence[int],
pos_array: Sequence[int],
seq_ids: Sequence[Sequence[int]],
logits_array: Sequence[bool]
)
llama_cpp/llama_embedding.py (around line 262) already calls add_sequence correctly with the four-arg shape — the call site in Llama.embed() was apparently missed during the LlamaBatch.add_sequence refactor.
Environment and Context
- Hardware: x86_64, NVIDIA GeForce RTX 4090
- OS: Windows 10 22H2
- Python 3.12.9
- llama-cpp-python 0.3.36 (CUDA 12.8 prebuilt wheel)
$ python --version
Python 3.12.9
$ pip show llama-cpp-python | findstr Version
Version: 0.3.36
Failure Information (for bugs)
This is a clean regression — LlamaBatch.add_sequence was refactored from a 3-arg signature to a 4-arg one, and the call sites were updated everywhere except in Llama.embed(). llama_embedding.py shows what the new shape should look like for the embedding code path.
Steps to Reproduce
from llama_cpp import Llama
m = Llama(model_path="path/to/model.gguf", embeddings=True)
m.embed("hello")
Result:
TypeError: LlamaBatch.add_sequence() missing 1 required positional argument: 'logits_array'
Failure Logs
Traceback (most recent call last):
File "...\Lib\site-packages\llama_cpp\llama.py", line 1678, in embed
self._batch.add_sequence(tokens, p_batch, logits_all)
TypeError: LlamaBatch.add_sequence() missing 1 required positional argument: 'logits_array'
Suggested fix
Mirror the call shape already used in llama_cpp/llama_embedding.py:
# In llama.py Llama.embed(), replace:
self._batch.add_sequence(tokens, p_batch, logits_all)
# With something like:
self._batch.add_sequence(
token_array=tokens,
pos_array=list(range(len(tokens))),
seq_ids=[p_batch],
logits_array=[True] * len(tokens) if logits_all else [False] * (len(tokens) - 1) + [True],
)
Workaround
Monkey-patching LlamaBatch.add_sequence to detect 3-arg legacy calls and synthesize the missing pos_array works as a stopgap. Hit while running Tencent's HY-Motion text-to-motion model, whose text encoder uses Llama.embed() against GGUF Qwen3 weights.
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.5k
- Avg merge
- 23m
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from abetlen/llama-cpp-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2352 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2210 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2145 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
abetlen/llama-cpp-python#2135 · 4 reactions ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2096 · 1 comment ·
All issues in abetlen/llama-cpp-python
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100