Scalable Pod IP matches in listener

Title: Scalable Pod IP matches in listener

Description: In Istio, we setup listeners with bind_to_port=false and use original destination redirection to match them. So we, roughly, have a Listener per kubernetes Service, matching on the Service IP. This works well enough.

The problem we have is traffic not to services. In Istio today, we support this only if there is a headless service associated with the pod. A headless service is more or less just a way to expose the backends via DNS, compared to a normal service which has VIP. From Envoy's perspective, they requests look the same. We only support headless services, not all direct-to-pod, since our approach is terrible from a scale perspective. What we currently do is just like services, we have a listener per pod.

This scales quite poorly. Pods are large in number and churning often. Listeners are large, and we haven't fully adopted delta XDS to send incremental updates for them.

Our current model is really not viable. We see for customers with modest headless services (~250 pods) have LDS payloads of 10mb+. Along with these churning frequently causing full pushes of the entire 10mb, this ends up being pretty terrible

We would like to improve on this state. Options discussed below:

1. Adopt Delta XDS.

No Envoy changes needed, this is a pure Istio side change. While we are working on this (for an number of reasons), this still doesn't fully solve the problem. Envoy still needs to full massive set of listeners, so its only an incremental improvement

2. Use AdditionalListeners

Instead of N listeners for a service, we can have 1 listener with N additional addresses. This almost works well, but a few problems:

• Changing any IP of them causes a drain for all connections, even to other IPs (I am 95% sure, want to double check though)
• There is one "primary" IP. This means if the one we pick happens to go away we have to shuffle things around. This is actually fine, as long as envoy handles this gracefully

3. Move to 1 big listener and filter chains

This gives us more flexibility, as we can just stick a list of IPs. This would mostly fix this problem. However, it contradicts (1). If we move to a single giant listener, we lose the ability to incrementally update things.

A theoretical new FilterChainDS could help this. Its not clear how this would interact with the new tree matcher, though.