grafana/loki

Does loki support migrating from one s3 cluster to another

Open

#10,711 opened on Sep 26, 2023

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (28,187 stars) (3,997 forks)batch import
help wantedtype/question

Description

I have a requirement to switch from cluster 1 of s3 to cluster 2 of s3 to ensure that the data of cluster 1 and cluster 2 can be read simultaneously.

From the perspective of configuration format, it seems that I support writing one s3 cluster. How should I write the configuration, or loki does not support writing two s3 clusters at the same time

Contributor guide

Tech stack
goaws
Domain
backend
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
needs maintainer response
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
Familiarity with Loki's storage configTwo S3 clusters
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
To answer this issue, first check Loki's official documentation on storage configuration, especially the 'storage' section in the config file. Look for parameters that allow multiple S3 endpoints or backends. Then, examine the source code in `pkg/storage/` to see if there is support for multiple S3 clusters or if a feature request exists. Also, search for related issues or PRs in the repository for 'multi cluster S3' or 'migrate S3'.
Does loki support migrating from one s3 cluster to another · grafana/loki#10711 | Good First Issue