[BUG] Message#to_h drops the provider-reported cost
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 67/100
Research direction
Start by reading Message#to_h, Message.new, and Tokens#to_h to trace how reported costs are represented; the issue also points to the existing round-trip work in commit 08035557. Use the provided OpenRouter cassette and reproduction to check serialization and restoration through Message.new or Chat#messages=. Done means the restored message retains the provider-reported cost, with coverage for the nil case and the documented multi-attempt behavior.
Written by the indexing model from the issue text.
Description
Basic checks
- I searched existing issues - this hasn't been reported
- I can reproduce this consistently
- This is a RubyLLM bug, not my application code
What's broken?
A response from a provider that reports its own price loses that price when it is serialized. Message.new(message.to_h) and Chat#messages= rebuild a message whose cost.total is a registry estimate, or nil when the registry has no pricing for the model.
Message.new accepts reported_cost: and Cost#total prefers it (e08033af). Message#to_h never emits it: it merges tokens.to_h, which leaves the reported cost to its own reader by design, and it emits :cost only when a cost was supplied explicitly. So the constructor understands a key that to_h does not produce. This is the same gap as #1001 (attachments) and #1035 (signature-only thinking) against the round trip 08035557 set up.
Affected: plain-Ruby transcripts exported with to_h and restored with Message.new or Chat#messages=, for every provider with a reported cost (OpenRouter, xAI). Nothing is raised. Rails persistence is not affected: ruby_llm_usages.total_cost keeps the amount.
How to reproduce
From the repo root on main, with no API key. It replays a response recorded in the repo's own cassette:
require 'bundler/setup'
require 'ruby_llm'
require 'webmock'
require 'yaml'
cassette = 'spec/fixtures/vcr_cassettes/chat_web_search_with_openrouter_openai_gpt-5_2_' \
'searches_transparently_returning_citations_and_usage_counters.yml'
body = YAML.load_file(cassette).dig('http_interactions', 0, 'response', 'body', 'string')
WebMock.enable!
WebMock.stub_request(:post, 'https://openrouter.ai/api/v1/chat/completions')
.to_return(body: body, headers: { 'Content-Type' => 'application/json' })
RubyLLM.configure { |config| config.openrouter_api_key = 'test' }
chat = RubyLLM.chat(model: 'openai/gpt-5.2', provider: :openrouter)
response = chat.ask('Search the web: what is the latest stable Ruby version? Cite your source.')
transcript = JSON.generate(chat.messages.map(&:to_h))
restored = RubyLLM.chat(model: 'openai/gpt-5.2', provider: :openrouter)
restored.messages = JSON.parse(transcript, symbolize_names: true)
puts "OpenRouter billed (usage.cost): #{JSON.parse(body).dig('usage', 'cost')}"
puts "response.cost.total: #{response.cost.total}"
puts "response.to_h[:reported_cost]: #{response.to_h[:reported_cost].inspect}"
puts "restored message cost.total: #{restored.messages.last.cost.total}"
Expected behavior
The restored message keeps the amount the provider billed:
restored message cost.total: 0.02199205
What actually happened
OpenRouter billed (usage.cost): 0.02199205
response.cost.total: 0.02199205
response.to_h[:reported_cost]: nil
restored message cost.total: 0.01199205
The restored cost is the registry price of the tokens. Here it misses the web search fee, 45% of the bill.
I have a small local fix with specs: Message#to_h emits reported_cost: tokens.reported_cost, omitted when nil. Tokens#to_h stays as documented.
@crmne, one question:
- A message with several attempts serializes the sum of the costs its attempts reported, which is what
message.tokens.reported_costreturns today. That is exact when every billed attempt reported a cost. When one attempt was priced from the registry and another reported its cost, the restored total is the reported part only (in a constructed case: 0.1239 before serializing, 0.123 restored with the fix, 0.0195 restored today). I recommend accepting that: it matchesCost.new(tokens: message.tokens)today, and a flatto_hcannot carry per-attempt costs. The alternative is to omitreported_costunless every attempt reported one. Which do you prefer?
Environment
- Ruby version: ruby 3.3.5
- RubyLLM version:
mainatfbef9a47 - Provider: OpenRouter (any provider with a reported cost)
- OS: macOS
- Dominant language
- Ruby
- Stars
- 4.4k
- Forks
- 510
- Avg merge
- 13h 57m
- Merged PRs (30d)
- 49
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from crmne/ruby_llm
-
[BUG] Chat replies are lost when a usage row cannot be linked to its messagePossibly taken @yorzi claimed this 1 day ago. Openbug
Difficulty 3/5 1-2 days Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
[BUG] Converted attachments are not uploaded again after a provider deletes the filePossibly taken @yorzi claimed this 1 day ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Maintainers usually reply within 1 day
Similar issues
-
Local evaluation buckets percentage splits with the server key, so results differ from FlagsmithOpen
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Add PrestashopOpenrequest
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
endoflife-date/endoflife.date#11303 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
FreeTubeApp/FreeTube-Docs#245 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day