Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[BUG] Chat replies are lost when a usage row cannot be linked to its message

Closed
  • Is something wrong?
#1,048 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

A pull request for this has already been merged.

  • #1049 by @yorzi — merged

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
postgresql, rails, ruby, sqlite
Domain
backend, databases

Research direction

Read lib/ruby_llm/active_record/chat_methods.rb around persist_message_completion and link_usage_entries, then review the existing usage-write handling referenced in the issue. Run the reproduction spec described in the issue and add regression coverage for a failed usage-row link. Done means the reply is returned and saved, a warning is logged, and the usage row remains in the ledger without a message, including on PostgreSQL.

Written by the indexing model from the issue text.

Description

bug
Basic checks
  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code
What's broken?

On an acts_as_chat record, ask raises and the reply is rolled back when the usage row of that reply cannot be linked to its message. The provider has already answered and billed. The chat is left with a blank assistant message, and the row stays in the ledger with no message.

persist_message_completion saves the reply in one transaction, and its last step is link_usage_entries (lib/ruby_llm/active_record/chat_methods.rb:743), which runs record&.update!(message: @message) with no savepoint and no rescue. Any failure of that UPDATE rolls back the reply's own transaction and reaches the caller. The blank placeholder stays, because cleanup_after_failure only runs for COMPLETION_ERRORS.

3647170a ("Keep chat replies when a usage row cannot be written") put the INSERT of the row in a savepoint and logs a failed write, "so a ledger write never breaks the conversation". The link is the second ledger write on the same path, and it was left as it was. AGENTS.md and the cost guide state the rule without that limit: "A ledger write never breaks the operation" and "Writing a row never breaks the operation that billed it."

Affected: every persisted chat. The unguarded link has been there since 2.0.0. The UPDATE has to fail after the INSERT succeeded, so this is rare. I reproduced it two ways:

  • a trigger that rejects the UPDATE, on SQLite and on PostgreSQL 18.4;
  • on PostgreSQL with no trigger: another session holds the new usage row while lock_timeout is set, and the link raises ActiveRecord::LockWaitTimeout.

A rescue alone would not be enough. On PostgreSQL a failed statement aborts the enclosing transaction, so I also checked a rescue without a savepoint there: ask returns Hi, and the commit silently drops the reply.

How to reproduce

From the repo root on main, with no API key. Save this as spec/usage_link_repro_spec.rb and run bundle exec rspec spec/usage_link_repro_spec.rb. The trigger rejects only the UPDATE that links a usage row, so the INSERT still succeeds.

require 'rails_helper'

RSpec.describe 'a usage row that cannot be linked to its reply' do
  include_context 'with configured RubyLLM'

  it 'still returns and saves the reply' do
    reply = { id: 'msg_1', type: 'message', role: 'assistant', model: model_for(:anthropic),
              content: [{ type: 'text', text: 'Hi' }], stop_reason: 'end_turn',
              usage: { input_tokens: 9, output_tokens: 2 } }
    stub_request(:post, 'https://api.anthropic.com/v1/messages')
      .to_return(status: 200, body: reply.to_json, headers: { 'Content-Type' => 'application/json' })
    connection = ActiveRecord::Base.connection
    connection.execute(<<~SQL)
      CREATE TRIGGER reject_usage_link BEFORE UPDATE OF message_id ON ruby_llm_usages
      BEGIN SELECT RAISE(ABORT, 'usage link rejected'); END
    SQL
    chat = Chat.create!(model: model_for(:anthropic))

    outcome = begin
      chat.ask('Hello').content
    rescue StandardError => e
      "#{e.class}: #{e.message}"
    end

    puts "ask:      #{outcome}"
    puts "messages: #{chat.reload.messages.order(:id).pluck(:role, :content).inspect}"
    puts "usages:   #{chat.ruby_llm_usages.pluck(:status, :output_tokens, :message_id).inspect}"
    expect(outcome).to eq('Hi')
  ensure
    connection&.execute('DROP TRIGGER IF EXISTS reject_usage_link')
  end
end
Expected behavior

The reply is returned and saved, a warning is logged, and the row stays in the ledger with its chat, without a message:

ask:      Hi
messages: [["user", "Hello"], ["assistant", "Hi"]]
usages:   [["succeeded", 2, nil]]
What actually happened
ask:      ActiveRecord::StatementInvalid: SQLite3::ConstraintException: usage link rejected
messages: [["user", "Hello"], ["assistant", ""]]
usages:   [["succeeded", 2, nil]]

On PostgreSQL 18.4, with the lock timeout and no trigger, the same two rows and:

ask:      ActiveRecord::LockWaitTimeout: PG::LockNotAvailable: ERROR:  canceling statement due to lock timeout
Environment
  • Ruby version: ruby 3.3.5
  • RubyLLM version: 2.1.0 (main at b8b939d3, the commit tagged v2.1.0)
  • Rails version: 8.1.3.1
  • Database: SQLite 3.53.2 (the dummy app), and PostgreSQL 18.4 for the second check
  • Provider: Anthropic, stubbed with WebMock. No live requests were made.
  • OS: macOS
Dominant language
Ruby
Stars
4.5k
Forks
523
Avg merge
9h 26m
Merged PRs (30d)
213

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from crmne/ruby_llm

All issues in crmne/ruby_llm

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.