No grpc stats present for mirrored traffic

Title: Grpc stats not present for shadow/mirrored traffic

Description:

What issue is being seen? Describe what should be happening instead of the bug, for example: Envoy should not crash, the expected value isn't returned, etc.

I don't see grpc stats present for mirrored traffic against my 'per instance' clusters (see the Config section for more details). I only see http level stats for these 'per instance' clusters.

Config:

Include the config used to configure Envoy.

Envoy config

1. There is one 'main cluster' containing all endpoints for all our services
2. For each service instance/box we have a 'per instance cluster' that only contains the endpoints for that instance/box

Routing

1. We have one route for all incoming traffic that goes to our main cluster

Validation My project validates newly spun up instances before their endpoints are added to this main cluster. This is done by adding a mirror policy that mirrors incoming traffic on the main route to the per instance cluster for these newly spun up instances. Once the instance is validated its endpoints are added to the main cluster.

Listener config We have grpc stats enabled using the HttpFilter

HttpFilter
        .newBuilder()
        .setName("envoy.filters.http.grpc_stats")
        .setTypedConfig(Any.pack(FilterConfig.newBuilder().build()))
        .build(),

Repro steps:

Include sample requests, environment, etc. All data and inputs required to reproduce the bug.

Route mirror traffic to a newly spun up instance, observe no grpc stats for that per instance cluster when curling stats from envoy's stats api.

Notes/Theories: I looked up the Envoy source code and I see that the implementation for sending mirrored traffic uses the HttpAsyncClient only, the GrpcAsyncClient is not used currently, so this could be the cause of the metrics being missing. My other theory is that mirrored traffic is not going through the http listener so it does not pass the the grpc_stats filter and collect those metrics.