Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[BUG] Message#to_h drops the provider-reported cost

Closed Beginner friendly
#1,045 1 comment 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
67/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
ruby
Domain
backend

Research direction

Start by reading Message#to_h, Message.new, and Tokens#to_h to trace how reported costs are represented; the issue also points to the existing round-trip work in commit 08035557. Use the provided OpenRouter cassette and reproduction to check serialization and restoration through Message.new or Chat#messages=. Done means the restored message retains the provider-reported cost, with coverage for the nil case and the documented multi-attempt behavior.

Written by the indexing model from the issue text.

Description

bug
Basic checks
  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code
What's broken?

A response from a provider that reports its own price loses that price when it is serialized. Message.new(message.to_h) and Chat#messages= rebuild a message whose cost.total is a registry estimate, or nil when the registry has no pricing for the model.

Message.new accepts reported_cost: and Cost#total prefers it (e08033af). Message#to_h never emits it: it merges tokens.to_h, which leaves the reported cost to its own reader by design, and it emits :cost only when a cost was supplied explicitly. So the constructor understands a key that to_h does not produce. This is the same gap as #1001 (attachments) and #1035 (signature-only thinking) against the round trip 08035557 set up.

Affected: plain-Ruby transcripts exported with to_h and restored with Message.new or Chat#messages=, for every provider with a reported cost (OpenRouter, xAI). Nothing is raised. Rails persistence is not affected: ruby_llm_usages.total_cost keeps the amount.

How to reproduce

From the repo root on main, with no API key. It replays a response recorded in the repo's own cassette:

require 'bundler/setup'
require 'ruby_llm'
require 'webmock'
require 'yaml'

cassette = 'spec/fixtures/vcr_cassettes/chat_web_search_with_openrouter_openai_gpt-5_2_' \
           'searches_transparently_returning_citations_and_usage_counters.yml'
body = YAML.load_file(cassette).dig('http_interactions', 0, 'response', 'body', 'string')

WebMock.enable!
WebMock.stub_request(:post, 'https://openrouter.ai/api/v1/chat/completions')
       .to_return(body: body, headers: { 'Content-Type' => 'application/json' })
RubyLLM.configure { |config| config.openrouter_api_key = 'test' }

chat = RubyLLM.chat(model: 'openai/gpt-5.2', provider: :openrouter)
response = chat.ask('Search the web: what is the latest stable Ruby version? Cite your source.')

transcript = JSON.generate(chat.messages.map(&:to_h))
restored = RubyLLM.chat(model: 'openai/gpt-5.2', provider: :openrouter)
restored.messages = JSON.parse(transcript, symbolize_names: true)

puts "OpenRouter billed (usage.cost):   #{JSON.parse(body).dig('usage', 'cost')}"
puts "response.cost.total:              #{response.cost.total}"
puts "response.to_h[:reported_cost]:    #{response.to_h[:reported_cost].inspect}"
puts "restored message cost.total:      #{restored.messages.last.cost.total}"
Expected behavior

The restored message keeps the amount the provider billed:

restored message cost.total:      0.02199205
What actually happened
OpenRouter billed (usage.cost):   0.02199205
response.cost.total:              0.02199205
response.to_h[:reported_cost]:    nil
restored message cost.total:      0.01199205

The restored cost is the registry price of the tokens. Here it misses the web search fee, 45% of the bill.

I have a small local fix with specs: Message#to_h emits reported_cost: tokens.reported_cost, omitted when nil. Tokens#to_h stays as documented.

@crmne, one question:

  1. A message with several attempts serializes the sum of the costs its attempts reported, which is what message.tokens.reported_cost returns today. That is exact when every billed attempt reported a cost. When one attempt was priced from the registry and another reported its cost, the restored total is the reported part only (in a constructed case: 0.1239 before serializing, 0.123 restored with the fix, 0.0195 restored today). I recommend accepting that: it matches Cost.new(tokens: message.tokens) today, and a flat to_h cannot carry per-attempt costs. The alternative is to omit reported_cost unless every attempt reported one. Which do you prefer?
Environment
  • Ruby version: ruby 3.3.5
  • RubyLLM version: main at fbef9a47
  • Provider: OpenRouter (any provider with a reported cost)
  • OS: macOS
Dominant language
Ruby
Stars
4.4k
Forks
510
Avg merge
13h 57m
Merged PRs (30d)
49

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from crmne/ruby_llm

All issues in crmne/ruby_llm

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.