Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

bottomless: finished S3 upload tasks are never reaped, so sqld memory grows by ~10 KB per WAL batch

未关闭 适合新手
  • 哪里不对？
#2,298 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

@pavanmanishd 已经在做这个了。

开始于 2026年10月22日。

  • #2299 来自 @pavanmanishd —— 未关闭

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
83/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
aws, docker, rust
领域
backend, databases

调研方向

从 bottomless/src/replicator.rs 开始，跟踪 Replicator::with_options 中的 S3 上传循环；阅读 JoinSet 在接收请求时如何处理已完成的任务。Issue 提议在该循环内回收已完成的任务。通过复现持续的 WAL 上传，并检查内存不再随每个批次增长，来确认这项更改。

由索引模型根据 Issue 内容生成。

描述

Summary

With bottomless replication enabled, sqld's memory grows linearly with the number of WAL batches it uploads and never comes back down. Every uploaded batch leaves about 10 KB behind. A database with a small but steady write stream (one write per second, default 1 s batch interval) grows by roughly 80–130 MB per day until the process is restarted or OOM-killed.

This may be the same report as #1781. It is separate from the snapshot-upload defects in #2260, which does not change the WAL upload loop.

Cause

In bottomless/src/replicator.rs, Replicator::with_options spawns the S3 upload loop. The loop creates an inner JoinSet and spawns one task per upload request:

let mut join_set = JoinSet::new();
while let Some(req) = frames_inbox.recv().await {
    // ...
    join_set.spawn(async move { /* put_object, remove_file, update progress */ });
}
while join_set.join_next().await.is_some() {}

Nothing reads the results of finished tasks until frames_inbox closes, which happens only at shutdown. A JoinSet keeps every finished task (its allocation and output) until it is joined, so each uploaded batch stays in memory for the lifetime of the replicator. new_generation reuses the same replicator, so the set is not reset at checkpoints either.

Fix

Reap finished tasks inside the loop:

                 while let Some(req) = frames_inbox.recv().await {
+                    while join_set.try_join_next().is_some() {}
                     tracing::trace!("Received S3 upload request: {}", req.path);

Reproducer

Local Docker, ghcr.io/tursodatabase/libsql-server:361d082 (same code in d6c75af / v0.24.33), S3-compatible store (RustFS) on the same network.

sqld environment (both containers):

SQLD_ENABLE_BOTTOMLESS_REPLICATION=true
LIBSQL_BOTTOMLESS_ENDPOINT=http://s3:9000
LIBSQL_BOTTOMLESS_BUCKET=sqld-leak
LIBSQL_BOTTOMLESS_DATABASE_ID=<unique per container>
LIBSQL_BOTTOMLESS_BATCH_INTERVAL_SECS=1
SQLD_CHECKPOINT_INTERVAL_S=60
LIBSQL_DISABLE_METRICS=true

Load: 10 INSERT OR REPLACE statements per second per container over HTTP. Memory: anon from the container's memory.stat.

45 min, 10 writes/s anon growth, minute 16 → 38 per uploaded batch
361d082, batch every 1 s 29.2 → 43.1 MB 10.8 KB
d6c75af, batch every 1 s same slope 11.2 KB
361d082, batch every 60 s flattens —
361d082 + fix above, batch every 1 s 18.2 → 21.8 MB 2.8 KB (see below)
bottomless off 6.3 → 6.6 MB —

Growth tracks the number of uploaded batches, not the number of writes: the same write load with a 60 s batch interval stays nearly flat.

Remaining growth with the fix

With the fix, about 2.8 KB per batch still accumulates. A 15-minute rheaper heap profile (/profile/heap/enable) of the patched build shows the live Rust heap flat at ≈ 0.25 MB and only 0.17 MB of allocations never freed, with no growing call stack. So the remainder is not a Rust-side leak. It is glibc keeping freed memory in per-thread arenas: the same patched build with MALLOC_ARENA_MAX=2 stays flat at 13.5 MB from minute 18 to minute 41 (1,380 batches), while the patched build with default glibc settings rises 17.0 → 21.2 MB over the same window.

Environment

  • libsql-server 361d082 and d6c75af (official images), linux/arm64 under Docker Desktop; same growth pattern observed on linux/amd64 in production.
  • Allocator: glibc (system).
主要语言
C
星标
17.3k
派生
545
30 天内没有已合并 PR

环境准备

  • 提供 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

tursodatabase/libsql 的其他 Issue

查看 tursodatabase/libsql 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。