Tune batch add parallelism constant · ipfs/kubo#4299

(0 comments) (0 reactions) (0 assignees)Go (13,906 stars) (2,725 forks)batch import

help wantedstatus/deferredtopic/perf

説明

Specifically, tune the ParallelBatchCommits constant added in #4296. Right now, we just use NumCPU*2 but we could probably do something fancier (learn it? does the datastore matter? does the storage medium matter? how about the filesystem?).

We may never get to this but hey, if anyone wants to spend some time experimenting they're free to take a crack at it.

コントリビューターガイド

技術スタック: go
領域: performance
Issue 種別: performance
難度: 3
推定時間: over 1 week
活動状況: stale
明確さ: unclear
前提条件: GoIPFS conceptsperformance profilingunderstanding of batch commit
初心者向け度: 20
調査方針: Investigate the `ParallelBatchCommits` constant added in #4296. Experiment with different values and measure performance across various datastores, storage media, and filesystems. Consider implementing adaptive tuning based on runtime metrics. Refer to the codebase for how the constant is used in batch add operations.