[FEATURE] Continue Gemini long-decode responses as one generation
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start with the proposed protocol behavior and the long-decode guide linked in the issue; the payload names spec/ruby_llm/protocols/gemini/long_decode_spec.rb as the reproduction. Trace the existing Gemini completion and streaming paths to understand how continuation tokens, assembled output, and per-request usage should fit. Done means one assistant turn reaches the terminal answer with correct accounting and the described error and streaming behavior; confirm a real continuation before merge.
Written by the indexing model from the issue text.
Description
Scope check
- This is core LLM communication (not application logic)
- This benefits most users (not just my use case)
- This can't be solved in application code with current RubyLLM
- I read the Contributing Guide
Due diligence
- I searched existing issues
- I checked the documentation
What problem does this solve?
Gemini GenerateContent can return finishReason: CONTINUATION with an encrypted continuationToken, including a thinking-only hop with no visible answer. RubyLLM currently returns that hop as an unfinished assistant response and never requests the terminal answer.
This is a protocol feature request, not a report of a live failure. The attached local unit reproduction uses the response shape documented in Google's long-decode guide and a stubbed terminal response. Baseline: 76ba13b8.
Proposed solution
The proposed implementation keeps the original rendered request, sends only the latest token on each continuation, assembles one local assistant turn, and records usage separately for every physical request. It does not create a stored conversation or use a provider response-history cursor. Streams continue at clean hop boundaries; a failure after chunks have been delivered still raises instead of replaying them.
Why this belongs in RubyLLM
The protocol must consume its own continuation signals and account for physical requests while returning one local turn. Reimplementing that in application code would require bypassing the normal completion and streaming paths.
Design question: is generation-scoped encrypted continuation acceptable under the local-transcript policy? I recommend yes. Google's guide says "The encrypted continuation_token carries the model's state between requests, so nothing is stored on the server." The token lives only while completing one immutable request, and only the assembled turn enters the transcript.
Reproduction: the new spec/ruby_llm/protocols/gemini/long_decode_spec.rb has failing-before public-API examples for empty hops, output assembly and accounting. Nine of the original thirteen examples fail on unchanged upstream. The local change adds Rails persistence, replay, error and streaming guards as well.
Before merge, record an actual continuation and confirm the non-streaming first-hop request. The guide's prose says a non-streaming request above 32,768 tokens without a token returns 400, while its REST example sends 262,144, so the documentation example streams. No production compatibility or live-provider verification is claimed here.
- Dominant language
- Ruby
- Stars
- 4.5k
- Forks
- 515
- Avg merge
- 9h 31m
- Merged PRs (30d)
- 220
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from crmne/ruby_llm
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
crmne/ruby_llm#1244 · 1 reaction ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 55/100
crmne/ruby_llm#1262 · 1 reaction ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
crmne/ruby_llm#1261 · 1 reaction ·
Maintainers usually reply within 1 day
-
[FEATURE] Preserve native client-toolset identity and resultsPossibly taken @yorzi claimed this 1 day ago. Openenhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
crmne/ruby_llm#1245 · 1 reaction ·
Maintainers usually reply within 1 day
Similar issues
-
`github-events` aborts the repository scan when a closed pull's API payload has a null `head`Possibly taken @renyamamoto2026-cmyk claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
zerocracy/judges-action#2823 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws/aws-sdk-ruby#3460 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Maintainers usually reply within 17 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
riscv/riscv-unified-db#2755 ·
Maintainers usually reply within 1 day