envoyproxy/envoy

Listener drops all connections when per filter config updated with RDS

Open

#17,109 opened on Jun 23, 2021

View on GitHub
 (20 comments) (1 reaction) (0 assignees)C++ (5,373 forks)batch import
area/perfarea/xdshelp wantedinvestigate

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (Avg merge 8d) (378 merged PRs in 30d)

Description

Title: Listener drops all connections when per filter config updated with RDS

Description: Having simple working setup:

static_resources {
  listeners {
    name: "listener"
    address {
      socket_address {
        address: "0.0.0.0"
        port_value: 8080
      }
    }
    filter_chains {
      filters {
        name: "envoy.http_connection_manager"
        typed_config {
          [type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager] {
            stat_prefix: "ingress_http"
            rds {
              config_source {
                ads {
                }
                initial_fetch_timeout {
                  seconds: 3
                }
                resource_api_version: V3
              }
              route_config_name: "xds-config-backend"
            }
            http_filters {
              name: "dropbox.envoy.waf"
              typed_config {
                [type.googleapis.com/dropbox.envoy_dbx.WafConfig] {
                }
              }
            }
            http_filters {
              name: "envoy.filters.http.router"
            }
// ... less important... //

Everything works well, including updates sent by XDS to Envoy while there is no "per vhost filter config overrides" - envoy successfully and without any impact to traffic able to update route configuration.

However, if route configuration contains per virtual host overrides, like

resources: <
  [type.googleapis.com/envoy.config.route.v3.RouteConfiguration]: <
    name: "xds-config-backend"
    virtual_hosts: <
      name: "test1"
      domains: "test1"
      routes: <
        match: <
          prefix: "/test1"
        >
        route: <
          cluster: "test_cluster"
        >
      >
      typed_per_filter_config: <
        key: "dropbox.envoy.waf"
        value: <
          [type.googleapis.com/dropbox.envoy_dbx.WafConfig]: <
          >
        >
      >
    >

Listener re-establishing all connections (downstream_cx_active gauge):

To me it seems like listener re-creates HttpConnectionManager and it somehow causes old connections to close.

Is it expected behavior? Is there some parameter that we can adjust? Any other ideas?

I'd be happy to provide more details, if needed.

Thanks

Contributor guide