Allow Setting Both Memory Requests and Limits for HdfsCluster Pods

Open
#625 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
kubernetes, rust
Domain
infrastructure

Research direction

Locate the HdfsCluster CRD definition and the resource defaulting or reconciliation path that builds pod resources. Check how the current memory limit is represented and propagated, then verify the schema and behavior for separate request and limit values. Done means explicit requests are honored while omitted requests retain the existing default behavior.

Written by the indexing model from the issue text.

Description

Current Behavior

The HdfsCluster Custom Resource Definition (CRD) currently only allows specifying memory.limit under HdfsCluster.spec.nameNodes.config.resources.memory. This configuration automatically sets memory.request equal to memory.limit for pods.

Problem

This behavior is problematic for CI platforms with limited memory resources. Forcing the memory request to match the limit can lead to resource scheduling issues, making it difficult to run HdfsCluster pods in resource-constrained environments.

Desired Behavior

Allow users to independently set both memory.request and memory.limit under HdfsCluster.spec.nameNodes.config.resources. For example:

spec:
  nameNodes:
    config:
      resources:
        memory:
          request: "512Mi"
          limit: "1Gi"

This flexibility ensures that pods can be scheduled even on CI platforms with tight resource constraints.

Proposed Solution
  • Update the HdfsCluster CRD to support separate request and limit fields for memory under resources.
  • Ensure backward compatibility by defaulting request to the same value as limit if request is not explicitly set.
Impact

This change would make the HdfsCluster CRD more usable in diverse environments, including CI platforms with restricted memory resources.

Additional Context

This enhancement would enable more efficient use of resources while maintaining compatibility with existing configurations.

Dominant language
Rust
Stars
53
Forks
9
Avg merge
1d 13h
Merged PRs (30d)
10

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stackabletech/hdfs-operator

All issues in stackabletech/hdfs-operator

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.