Memory management cgroup diagram does not show which config value sizes each region
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- documentation
Research direction
Open the existing cgroup diagram in the linked memory_management section and review how its regions are currently grouped. Add the three configuration values described by the issue as diagram inputs, showing which regions each sizes, then render the documentation and verify that the off-heap and memory-overhead relationships are visible without relying on prose.
Written by the indexing model from the issue text.
Description
The cgroup diagram in memory_management, added in #6014, groups everything the container counts by who accounts for it. It does not say which configuration value sizes each region.
That leaves the two facts the section goes on to state as prose only:
spark.memory.offHeap.sizeis part of the pod limit, not extra headroom on top of it.spark.executor.memoryOverheadis the only slack in the container.
Both are properties of how the configuration maps onto the regions, which is exactly the kind of
thing a diagram carries better than a sentence. The mapping is also uneven in a way that is easy to
get wrong: spark.memory.offHeap.size alone sizes three of the regions, including Comet's native
reservations, which are neither off-heap in Spark's sense nor allocated by the JVM, while
spark.executor.memoryOverhead sizes none of them.
Adding the three configuration values as inputs to the existing diagram would make that visible
without introducing a third diagram to the page.
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 377
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 244
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion-comet
-
area:ci enhancement requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6078 ·
-
area:ci bug priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6060 ·
-
enhancement
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
apache/datafusion-comet#5861 ·
-
requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/datafusion-comet#5661 ·
-
requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/datafusion-comet#5601 ·
All issues in apache/datafusion-comet
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
ergoplatform/ergodocs#614 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
[VL] madvise(WILLNEED) call fails in MmapFileStream because of wrong calculation of fetching length Openbug triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
chipsalliance/rocket-chip#3831 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100