simonw/llm
View on GitHubEmbeddings need to handle truncation automatically (maybe via extra options)
Open
#1,085 opened on May 25, 2025
embeddingsenhancementhelp wanted
Repository metrics
- Stars
- (12,006 stars)
- PR merge metrics
- (PR metrics pending)
Description
I frustrating thing about embeddings right now is this:
cd llm
llm embed-multi --files docs/ '*.md' -d /tmp/docs.db docs
This fails with a messy error:
...
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/openai/resources/embeddings.py", line 128, in create
return self._post(
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/openai/_base_client.py", line 1242, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/openai/_base_client.py", line 919, in request
return self._request(
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/openai/_base_client.py", line 1023, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens, however you requested 17143 tokens (17143 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.", 'type': 'invalid_request_error', 'param': None, 'code': None}}
That error reformatted:
{
"error": {
"message": "This model's maximum context length is 8192 tokens, however you requested 17143 tokens (17143 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.",
"type": "invalid_request_error",
"param": null,
"code": null
}
}
NO embeddings are stored in the /tmp/docs.db database. The entire operation just fails, with no obvious way to recover.