Make `ext_authz` failure-mode-allow header configurable

Title: Make ext_authz failure-mode-allow header configurable

Description:

Describe the desired behavior, what scenario it enables and how it would be used.

ext_authz failure-mode-allow header leaks implementation to potentially untrusted upstreams. It is perhaps unadvisable to use ext_authz with untrusted upstreams, and failure-mode-allow can be disabled.

The addition of the x-envoy-auth-failure-mode-allowed leaks auth implementation and/or security details to potentially untrusted upstreams.

The feature was added in https://github.com/envoyproxy/envoy/pull/26326. A decision was made to not make this permanently configurable here. No method of removing this header is provided without adding a sanitizing filter after all extauth filters.

Reproduction: Run envoy with an ext_authz pointing to a non-existent endpoint and hit an echo server to see the headers received by the upstream when a failure occurs in authz.

Script must be run on Linux and requires node installed on $PATH. Change the envoy download path and/or use docker if desired. Any echo upstream can be used; a node server was provided for completeness.

#!/bin/bash

# Add envoy config yaml
cat > ./envoy.yaml <<EOF
    static_resources:
      listeners:
        - address:
            socket_address:
              address: 0.0.0.0
              port_value: 19000
          filter_chains:
            - filters:
                - name: envoy.filters.network.http_connection_manager
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                    codec_type: auto
                    stat_prefix: ingress_http
                    route_config:
                      name: local_route
                      virtual_hosts:
                        - name: backend
                          domains:
                            - "*"
                          routes:
                            - match:
                                prefix: "/"
                              route:
                                cluster: service
                    http_filters:
                      - name: envoy.filters.http.router
                        typed_config:
                          "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
        - address:
            socket_address:
              address: 0.0.0.0
              port_value: 8082
          filter_chains:
            - filters:
                - name: envoy.filters.network.http_connection_manager
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                    codec_type: auto
                    stat_prefix: ingress_http
                    route_config:
                      name: local_route
                      virtual_hosts:
                        - name: backend
                          domains:
                            - "*"
                          routes:
                            - match:
                                prefix: "/"
                              route:
                                cluster: test-node-server
                    http_filters:
                      - name: envoy.filters.http.ext_authz
                        typedConfig:
                          '@type': type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
                          failureModeAllow: true
                          grpcService:
                            envoyGrpc:
                              clusterName: bad-extauth
                            timeout: 0.200s
                          metadataContextNamespaces:
                          - envoy.filters.http.jwt_authn
                          transportApiVersion: V3
                      - name: envoy.filters.http.router
                        typed_config:
                          "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      clusters:
        - name: service
          connect_timeout: 0.25s
          type: strict_dns
          lb_policy: round_robin
          load_assignment:
            cluster_name: service
            endpoints:
              - lb_endpoints:
                  - endpoint:
                      address:
                        socket_address:
                          address: 127.0.0.1
                          port_value: 19001
        - name: test-node-server
          connect_timeout: 0.25s
          type: strict_dns
          lb_policy: round_robin
          load_assignment:
            cluster_name: test-node-server
            endpoints:
              - lb_endpoints:
                  - endpoint:
                      address:
                        socket_address:
                          address: 127.0.0.1
                          port_value: 4000
        - name: bad-extauth
          connect_timeout: 0.25s
          type: strict_dns
          lb_policy: round_robin
          load_assignment:
            cluster_name: bad-extauth
            endpoints:
              - lb_endpoints:
                  - endpoint:
                      address:
                        socket_address:
                          address: does.not.exist
                          port_value: 4000
    admin:
      access_log_path: "/dev/null"
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 19001
EOF

# Add node server script
cat > ./node-server.js <<'EOF'
let http = require("http")

let host = '0.0.0.0'
let port = 4000

let server = http.createServer((req, res) => {
    res.writeHead(200)
    res.end(JSON.stringify(req.headers))
})

server.listen(port, host, () => {
    console.log(`Server is running on http://${host}:${port}`)
})
EOF

# Get envoy 1.26.2
curl -LO "https://github.com/envoyproxy/envoy/releases/download/v1.26.2/envoy-x86_64"
mv envoy-x86_64 envoy
chmod +x envoy

# Run our test upstream
node node-server.js &
NODE_SERVER_PID=$!

# Run envoy
./envoy --config-path ./envoy.yaml --log-level debug 1>./envoy.log 2>&1 &

# Wait for envoy to initialize
echo "waiting 2 seconds for Envoy to initialize"
sleep 2

# Make a request to see the echoed headers
curl http://localhost:8082/ -v

# Kill envoy
curl -X 'POST' http://localhost:19000/quitquitquit

# Kill node script
kill $NODE_SERVER_PID

From the output of this script we can see the header is received by the upstream x-envoy-auth-failure-mode-allowed: true

[optional Relevant Links:]

Any extra documentation required to understand the issue.

Implementing PR: https://github.com/envoyproxy/envoy/pull/26326 Comment where decision was made to not make this permanently configurable: https://github.com/envoyproxy/envoy/pull/26326#issuecomment-1492817386