UDP routing remains active after deleting UDPRoute

Hi. I have a Gateway with exactly one listener. This listener is of the protocol UDP and a UDPRoute references it to route the UDP traffic to a coredns service (similiar to the UDP Routing guide in the documentation). This works as expected, however, when I delete the UDPRoute again the routing remains active and I am still able to reach the coredns service using dig. Only the deletion of the Gateway itself stops the UDP routing.

On Gateways with more than a single UDP listener this issue does not occur.

Steps to reproduce:

1. Deploy envoy gateway to a cluster
2. Create a Gateway with a UDP listener and no other listeners on that Gateway
3. Create a UDPRoute to route traffic via that Gateway to the coredns service (like in the UDP Routing guide)
4. Use dig to confirm the proper functioning of the UDPRoute
5. Delete the UDPRoute
6. Find UDP routing still active

I use the envoy gateway v1.2.4 (envoy v1.32.2) on a v1.27 k8s cluster with the following config:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
  name: eg
  namespace: default
spec:
  gatewayClassName: eg
  listeners:
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: UDPRoute
      namespaces:
        from: Same
    name: coredns
    port: 5300
    protocol: UDP
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: UDPRoute
metadata:
  annotations:
  name: coredns
  namespace: default
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
    sectionName: coredns
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: coredns
      port: 53
      weight: 1
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: coredns
  name: coredns
  namespace: default
spec:
  ports:
  - name: udp-dns
    port: 53
    protocol: UDP
    targetPort: 53
  selector:
    app: coredns
  type: ClusterIP