Base backups are single-threaded and ignore concurrency settings
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- go, postgresql
- Domain
- cloud, databases, performance
Research direction
Start by tracing the plugin-barman-cloud base-backup upload path and compare it with WAL archiving's maxParallel handling. Determine how the data upload can honor a configurable parallel-job setting; done means the base backup phase supports parallel uploads without changing the existing default behavior.
Written by the indexing model from the issue text.
Description
The plugin-barman-cloud performs base backups using a single upload stream. Unlike WAL archiving (which supports maxParallel), there is no way to parallelize the data backup phase. This results in bottlenecks for large databases.
This option:
$ kubectl explain objectstore.spec.configuration.data.jobs
GROUP: barmancloud.cnpg.io
KIND: ObjectStore
VERSION: v1
FIELD: jobs <integer>
DESCRIPTION:
The number of parallel jobs to be used to upload the backup, defaults
to 2
is not applicable due to limitation (only for rsync):
https://github.com/EnterpriseDB/barman/blob/ff040cd1feee805da3e4662a187c583f8730dfab/docs/user_guide/configuration.rst?plain=1#L336
parallel_jobs
Controls the number of parallel workers used to copy files during backup or recovery.
It must be a positive integer. Default is1.
note::
Applies only whenbackup_method = rsync.
- Dominant language
- Go
- Stars
- 192
- Forks
- 75
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 17
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cloudnative-pg/plugin-barman-cloud
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
cloudnative-pg/plugin-barman-cloud#1104 · 4 reactions ·
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
-
Difficulty 3/5 1-2 days Newbie friendliness 65/100
cloudnative-pg/plugin-barman-cloud#1113 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 15/100
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
All issues in cloudnative-pg/plugin-barman-cloud
Similar issues
-
bug github_actions
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
registrystack/registry-stack#1393 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
JakeChampion/lang#10213 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
oasisprotocol/oasis-sdk#2523 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100