Add configurable HTTP timeouts and error callback to `get_prerendered_page_response`

Open Beginner friendly
#69 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
Half a day
Newbie friendliness
68/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
rails, ruby
Domain
backend

Research direction

Start at get_prerendered_page_response, where Net::HTTP is created and the existing before_render/after_render option pattern is used. Add the optional timeout settings and on_error callback described in the issue, preserving the nil fallback behavior; done means supplied values reach Net::HTTP and failures invoke the callback without changing the default behavior.

Written by the indexing model from the issue text.

Description

Problem

get_prerendered_page_response creates a Net::HTTP instance without setting open_timeout or read_timeout:

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true if url.scheme == 'https'
response = http.request(req)

This inherits Ruby's Net::HTTP defaults of 60 seconds for both connection and read timeouts. When the Prerender service is degraded (slow responses, gateway errors), each request holds the calling web server thread/process for up to 60 seconds before timing out.
In threaded application servers like Puma, this quickly saturates the worker pool — a single slow Prerender dependency can take down the entire application for all users, not just bot traffic.
The bare rescue => nil also means timeout errors are silently swallowed with no opportunity for the consuming application to log, alert, or take corrective action (e.g., circuit breaking).

Proposed Solution

1. Configurable timeouts via the options hash

Allow consumers to pass open_timeout and read_timeout through the existing options mechanism:

config.middleware.use Rack::Prerender,
  prerender_token: 'YOUR_TOKEN',
  open_timeout: 5,
  read_timeout: 10

Implementation in get_prerendered_page_response:

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true if url.scheme == 'https'
http.open_timeout = @options[:open_timeout] if @options[:open_timeout]
http.read_timeout = @options[:read_timeout] if @options[:read_timeout]
response = http.request(req)

When not provided, behavior is unchanged (Net::HTTP defaults apply). Fully backward-compatible.

2. Error callback via on_error option

Expose an optional on_error callback (consistent with the existing before_render / after_render pattern) so consuming applications have visibility into failures:

config.middleware.use Rack::Prerender,
  prerender_token: 'YOUR_TOKEN',
  open_timeout: 5,
  read_timeout: 10,
  on_error: Proc.new do |error, env|
    Rails.logger.warn("Prerender request failed: #{error.class} - #{error.message}")
  end

Implementation — replace the current bare rescue:

rescue => e
  @options[:on_error].call(e, env) if @options[:on_error]
  nil

The return value remains nil (falling through to @app.call), preserving existing behavior. The callback is purely for observability and consumer-side error handling.

Context

We experienced a production incident where Prerender.io degradation caused 30-60 second response times. With the default 60-second Net::HTTP timeout, Puma workers were held for the full duration of each request. This saturated the worker pool within minutes, causing the upstream reverse proxy (NGINX) to return 503s to all traffic — including non-bot requests that don't use Prerender at all.
Reducing the timeout to 5-10 seconds and having visibility into errors would have limited the blast radius significantly.

Alternatives Considered

  • Wrapping Net::HTTP.new: More invasive, breaks if the HTTP implementation changes.
  • Using Timeout.timeout: Dangerous in Ruby — can interrupt code at unpredictable points. Native Net::HTTP timeouts are the correct approach.
Dominant language
Ruby
Stars
358
Forks
87
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from prerender/prerender_rails

All issues in prerender/prerender_rails

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.