Goal

What I'm hoping to achieve is a discussion around how we can add automated performance testing into the lobsters testsuite.

Problem

I would like to look into fixing the performance regressions introduced in https://github.com/lobsters/lobsters/pull/1927 but there is currently no strategy other than manual testing and fixing related to performance issues in the lobsters code base.

I was thinking of writing some performance tests with large amounts of data like the following proof of concept I have:

require "rails_helper"

describe StoriesController do
  it "performs the show action quickly on a large amount of comments", :perf do
    user = create(:user)
    stories = create_list(:story, 120_000, user: user) do |story, _i|
      create_list(:comment, 6, user: user, story: story)
    end
    story = stories.sample

    expect { get :show, params: {id: story.short_id} }.to perform_under(200).ms
  end
end

The problem is that generating this large amount of data is very slow... on the order of 30+ minutes and counting just for this one test which I don't think would be acceptable for any workflow.

I know that you can export table statistics in sqlite (though I don't know how) but I think even exporting these statistics wouldn't be very helpful since they would only be useful for analyzing the query plans which as far as I know, there aren't any libraries or functions in rails that would allow you to easily verify that a query plan avoids table scans in sqlite. Checking the runtime of a query would still have to be tested against a large dataset.