Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Operator normalizes single-element grant arrays to strings, causing Helm upgrade conflicts

Open
  • Is something wrong?
#1,870 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
go, helm, kubernetes
Domain
databases, infrastructure

Research direction

Start by tracing the reconciliation logic that processes the users configuration and generates the ClickHouse ConfigMap; no specific file or test is identified in the issue. Reproduce the Helm deployment and upgrade sequence with one grant becoming multiple grants, then verify that reconciliation preserves the array format and the upgrade no longer reports a conflict.

Written by the indexing model from the issue text.

Description

The ClickHouse Operator appears to normalize single-element arrays in the spec.configuration.users.<username>/grants/query field to string values during reconciliation. This causes Helm upgrade conflicts when later adding more grants to the array.

Environment
  • ClickHouse Operator Version: 0.25.5
  • Kubernetes Version: (your version)
  • Helm Version: (your version)
  • Deployment Method: Helm chart
Steps to Reproduce
  1. Initial deployment with a single grant:

Deploy a CHI resource via Helm with one grant:

spec:
  configuration:
    users:
      metabase/grants/query:
        - "GRANT SELECT ON logs.*"
  1. Check the stored resource:
kubectl get chi clickhouse -n observability -o yaml | grep -A 5 "metabase/grants"

Expected: Array format

metabase/grants/query:
  - "GRANT SELECT ON logs.*"

Actual: String format (after operator reconciliation)

metabase/grants/query: "GRANT SELECT ON logs.*"
  1. Attempt to add more grants:

Update the Helm values to include multiple grants:

spec:
  configuration:
    users:
      metabase/grants/query:
        - "GRANT SELECT ON logs.*"
        - "GRANT SELECT ON system.processes"
        - "GRANT KILL QUERY ON *.*"
  1. Run Helm upgrade:
helm upgrade clickhouse ./chart -n observability

Result: Helm reports a conflict error:

Error: UPGRADE FAILED: conflict occurred while applying object observability/clickhouse 
clickhouse.altinity.com/v1, Kind=ClickHouseInstallation: 
Apply failed with 1 conflict: conflict with "helm" using clickhouse.altinity.com/v1: 
.spec.configuration.users.metabase/grants/query
Root Cause Analysis

The issue occurs because:

  1. The CRD uses x-kubernetes-preserve-unknown-fields: true for the users field
  2. When a single-element array is deployed, the operator normalizes it to a string during reconciliation
  3. When Helm tries to update from string to array, it detects a type incompatibility and reports a conflict
  4. Helm's three-way strategic merge cannot automatically resolve type changes (string → array)
Impact
  • Helm deployments fail when adding additional grants to users that initially had only one grant
  • Requires manual intervention (kubectl patch) to resolve
  • Inconsistent behavior: 2+ element arrays are preserved, but single-element arrays are converted
  • Breaks GitOps workflows where configuration is managed declaratively through Helm
Workaround

Option 1: Always use 2+ grants (add a harmless grant):

users:
  - name: metabase
    grants:
      - "GRANT SELECT ON logs.*"
      - "GRANT SHOW ON *.*"  # Dummy grant to prevent normalization

Option 2: Use kubectl patch to update grants:

kubectl patch chi clickhouse -n observability --type=json -p='[...]'
Expected Behavior

The operator should preserve the array format even for single-element arrays, to maintain consistency and avoid Helm conflicts.

Suggested Solutions
  1. Disable single-element array normalization for grant fields
  2. Add a configuration option to control normalization behavior
  3. Document this behavior prominently in the operator documentation
  4. Preserve original data types from the API server without modification
Additional Context

This behavior is particularly problematic for teams using:

  • Helm for infrastructure-as-code
  • GitOps workflows (ArgoCD, Flux)
  • Gradual permission expansion (starting with minimal grants)

The normalization might be intended for optimization, but it creates operational challenges in production environments where declarative configuration management is critical.

Related Code

The issue likely originates in the operator's reconciliation logic where it processes the users configuration and generates the ConfigMap for ClickHouse.

Dominant language
Go
Stars
2.6k
Forks
577
Avg merge
1d 6h
Merged PRs (30d)
4

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Altinity/clickhouse-operator

All issues in Altinity/clickhouse-operator

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.