Memory usage is too high for large tenant

Open
#67 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
azure, kubernetes, powershell

Research direction

Start with AzGovVizParallel.ps1 and reproduce the reported command using the large-tenant options, memory reporting, and the supplied environment variables. Compare memory around subscription processing and TenantSummary RoleAssignments processing. Done means the 115-subscription, 39,558-role-assignment workload no longer grows until it is OOMKilled.

Written by the indexing model from the issue text.

Description

AzGovViz version
6.7.2

CodeRunPlatform
Jenkins, on AKS, with azure-powershell runner image, version: '15.0.0-alpine-3.22'.
limitMemory:'12Gi', requestMemory:'3Gi', limitCpu:'2.5'

Describe the bug
Script memory usage is growing continuously, without important memory release, and finish by a OOMKilled, because it is using all the memory.

Despite a aggressive garbage collection settings.

First memory ramp end with "CustomDataCollection Subscriptions processing" end (16:59).
A secund memory ramp end after "processing (pre) TenantSummary RoleAssignments" (17:17).

Screenshots
Image

Additional context

Command line :
pwsh/AzGovVizParallel.ps1 -ManagementGroupId 'myMainMG' -OutputPath 'AzGovViz-Output' -HierarchyMapOnly -NoPIMEligibility -ShowMemoryUsage -LargeTenant -DoTranscript -CriticalMemoryUsage 70

Environment vars :
DOTNET_GCHeapHardLimitPercent=0x63
DOTNET_GCHeapCount=4

Landing zone insights :

  • 54 relevant Management Groups
  • 115 relevant Subscriptions
  • RoleAssignments (all 39558)
Dominant language
PowerShell
Stars
185
Forks
72
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Azure/Azure-Governance-Visualizer

All issues in Azure/Azure-Governance-Visualizer

Similar issues

More DevOps issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.