Function-Based Custom Rule with "Error" Severity Doesn’t Show Under -Severity Error

Reproduce the issue with Measure-AvoidCustomInvokeExpression.psm1 and Invoke-GetProcess.ps1 using PSScriptAnalyzer 1.23.0 and both severity filters. Start by tracing the custom-rule diagnostic severity through the analyzer's severity filtering entry point. Done means an Error diagnostic appears with -Severity Error while existing severity filtering continues to work.

I'm experiencing an issue with function-based custom rules. When a custom rule returns a severity of Error, the results don't appear under the -Severity Error filter. Instead, they show up only when filtering by warnings or when no filter is applied.

I couldn’t find any other reports of a similar issue. The closest I came across was #1237, but it seems unanswered or possibly overlooked.

The use-case here is for demonstration purposes.

Steps to reproduce

Custom Rule (Measure-AvoidCustomInvokeExpression.psm1)

function Measure-AvoidCustomInvokeExpression {
    [CmdletBinding()]
    [OutputType([Microsoft.Windows.Powershell.ScriptAnalyzer.Generic.DiagnosticRecord[]])]
    param (
        [Parameter(Mandatory = $true)]
        [ValidateNotNullOrEmpty()]
        [System.Management.Automation.Language.ScriptBlockAst] $ScriptBlockAst
    )
    process {
        $results = @()
        
        try {
            [ScriptBlock]$predicate = {
                param (
                    [System.Management.Automation.Language.Ast] $Ast
                )
                [bool]$returnValue = $false
                
                if ($Ast -is [System.Management.Automation.Language.CommandAst]) {
                    [System.Management.Automation.Language.CommandAst]$commandAst = $Ast
                    if ($commandAst.GetCommandName() -eq 'Invoke-Expression') {
                        $returnValue = $true
                    }
                }
                return $returnValue
            }
            
            [System.Management.Automation.Language.Ast[]]$asts = $ScriptBlockAst.FindAll($predicate, $true)
            if ($asts.Count -ne 0) {
                foreach ($ast in $asts) {
                    $result = New-Object `
                        -TypeName "Microsoft.Windows.Powershell.ScriptAnalyzer.Generic.DiagnosticRecord" `
                        -ArgumentList `
                        "Stop it!", 
                        $ast.Extent, 
                        "AvoidCustomInvokeExpression", 
                        Error, 
                        $null
                    
                    $results += $result
                }
            }
            return $results
        }
        catch {
            $PSCmdlet.ThrowTerminatingError($PSItem)
        }
    }
}

Export-ModuleMember -Function Measure-AvoidCustomInvokeExpression

Test Script (Invoke-GetProcess.ps1)

Invoke-Expression "Get-Process"

Expected behavior

Invoke-ScriptAnalyzer -Path ./Invoke-GetProcess.ps1 -CustomRulePath ./Measure-AvoidCustomInvokeExpression.psm1 -Severity Error

RuleName                            Severity     ScriptName Line  Message
--------                            --------     ---------- ----  -------
Custom Name                         Error        Invoke-Get 1     Stop it!
                                                 Process.ps
                                                 1

Actual behavior

Used with -Severity Error

Invoke-ScriptAnalyzer -Path ./Invoke-GetProcess.ps1 -CustomRulePath ./Measure-AvoidCustomInvokeExpression.psm1 -Severity Error

<nothing>

Used with -Severity Warning

Invoke-ScriptAnalyzer -Path ./Invoke-GetProcess.ps1 -CustomRulePath ./Measure-AvoidCustomInvokeExpression.psm1 -Severity Warning

RuleName                            Severity     ScriptName Line  Message
--------                            --------     ---------- ----  -------
Custom Name                         Error        Invoke-Get 1     Stop it!
                                                 Process.ps
                                                 1

Environment data

Windows 11

> $PSVersionTable
PSVersion                      5.1.22621.4391
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
BuildVersion                   10.0.22621.4391
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.23.0

Manjaro 6.6.63-1

> $PSVersionTable
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1-0-g5668713d3c906d63cd68e37d415206a95ac061d0
OS                             Manjaro Linux
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.23.0