[BUG] embed(array) returns one vector for many texts with Bedrock Titan behind an OpenAI-compatible gateway
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start with lib/ruby_llm/protocols/chat_completions/embeddings.rb, where the response rows are handled. Add validation that the number of returned embeddings matches the number of input texts, raising the appropriate RubyLLM::Error when it does not. Exercise the issue's array-embedding reproduction and verify that a complete response still returns one vector per text.
Written by the indexing model from the issue text.
Description
Basic checks
- I searched existing issues - this hasn't been reported
- I can reproduce this consistently
- This is a RubyLLM bug, not my application code
What's broken?
On OpenAI-compatible endpoints, RubyLLM.embed(array) returns whatever rows the server sends. It never checks that there is one row per input (embeddings.rb:23).
We hit this with Portkey in front of Amazon Titan, which takes one input per request. The gateway returns a single vector for the first text and drops the rest. RubyLLM returns that one vector for the whole array, without an error.
The index check from #1160 does not catch this: one row with index: 0 is valid. Switching to provider: :bedrock, which sends one request per text for Titan, doesn't help either: it only signs requests with SigV4, so it can't use a gateway bearer key.
How to reproduce
require "ruby_llm"
# An OpenAI-compatible gateway (Portkey) routing to Bedrock Titan,
# e.g. GATEWAY_MODEL="@<bedrock-slug>/amazon.titan-embed-text-v2:0"
RubyLLM.configure do |config|
config.openai_api_key = ENV.fetch("GATEWAY_API_KEY")
config.openai_api_base = ENV.fetch("GATEWAY_BASE_URL")
end
texts = %w[Ruby Python JavaScript]
embedding = RubyLLM.embed(texts, model: ENV.fetch("GATEWAY_MODEL"), provider: :openai, assume_model_exists: true)
puts "#{texts.size} texts -> #{embedding.vectors.size} vector(s)"
Expected behavior
3 texts -> 3 vector(s)
Or a RubyLLM::Error such as Expected 3 embeddings, got 1
What actually happened
3 texts -> 1 vector(s)
No error is raised. The returned vector is the first text's embedding
Environment
- RubyLLM version: 2.1.1
- Provider (OpenAI, Anthropic, etc.):
:openaivia a Portkey gateway, modelamazon.titan-embed-text-v2:0
- Dominant language
- Ruby
- Stars
- 4.5k
- Forks
- 523
- Avg merge
- 13h 49m
- Merged PRs (30d)
- 221
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from crmne/ruby_llm
-
[BUG] Message#parsed reads the commentary items of a Responses answer, unlike the OpenAI SDKPossibly taken @crmne claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
crmne/ruby_llm#1281 · 2 reactions ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
crmne/ruby_llm#1291 · 1 reaction ·
Maintainers usually reply within 1 day
-
[BUG] macOS code execution cannot load Homebrew-linked Ruby librariesPossibly taken @yorzi claimed this 3 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 35/100
crmne/ruby_llm#1278 · 2 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
[FEATURE] Continue Gemini long-decode responses as one generationPossibly taken @yorzi claimed this 6 days ago. Openenhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
crmne/ruby_llm#1246 · 1 reaction ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
rubyevents/rubyevents#2277 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 92/100
Maintainers usually reply within 2 days