Document GCS bucket retention policy impact on ClickHouse storage

Open Beginner friendly
#147 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
72/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Stale
Tech stack
clickhouse, google-cloud

Research direction

Read content/en/altinity-kb-integrations/altinity-kb-google-s3-gcs.md and review its existing GCS setup guidance. Add a warning explaining how retention policies block ClickHouse deletion during merges, mutations, and TTL expiration, and recommend using a bucket without a retention policy. Done means the storage-growth and operational-cost risk is clearly documented.

Written by the indexing model from the issue text.

Description

Problem

The current GCS documentation (content/en/altinity-kb-integrations/altinity-kb-google-s3-gcs.md) only covers basic setup (HMAC keys, endpoint configuration) but doesn't mention an important operational caveat.

Missing Information

When a GCS bucket has a retention policy enabled, objects cannot be deleted until the retention period expires. This causes issues with ClickHouse because it regularly deletes data during:

  • Part merges - old parts are removed after merge
  • Mutations - replaced data is deleted
  • TTL expiration - expired data is purged

If the bucket retention policy prevents deletion, old data accumulates and storage grows exponentially, leading to unexpected costs and potential operational issues.

Suggested Addition

Add a warning/note to the GCS documentation explaining:

  1. Bucket retention policies prevent ClickHouse from deleting obsolete data
  2. This causes continuous storage growth
  3. Recommendation to disable retention policy or use a bucket without one for ClickHouse storage

If this seems like a good enhancement, I'd be happy to take this up and submit a PR.

Dominant language
HTML
Stars
62
Forks
49
Avg merge
6m
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Altinity/altinityknowledgebase

All issues in Altinity/altinityknowledgebase

Similar issues

More Databases issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.