Memory leak from Redis due to missing key expirations

Open
#280 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
redis, ruby
Domain
databases

Research direction

Start by tracing where the Redis keys requeues-count and running are created and where the default 8-hour TTL is applied. Verify which queue keys receive expiration and inspect the linked code search for these keys; done means the affected keys no longer persist indefinitely and the behavior is covered by the project's existing tests.

Written by the indexing model from the issue text.

Description

Hi, we are using ci-queue for minitest and noticed that the memory usage in our Redis cluster grows slowly over time, despite using the default TTL of 8 hours. It appears that this is because not all keys have a corresponding expire call. For example, the requeues-count key is never expired. Same thing for the running key. Is there any reason that only certain keys are marked as expired?

Dominant language
Ruby
Stars
240
Forks
36
Avg merge
28m
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Shopify/ci-queue

All issues in Shopify/ci-queue

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.