envoyproxy/envoy

EDS requests sent after receiving SIGTERM

Open

#13,900 opened on Nov 4, 2020

View on GitHub
 (5 comments) (0 reactions) (0 assignees)C++ (27,997 stars) (5,373 forks)batch import
area/xdsbughelp wanted

Description

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via emailing envoy-security@googlegroups.com where the issue will be triaged appropriately.

Title: EDS requests sent after receiving SIGTERM

Description: When envoy is sent SIGTERM, it will start tearing down clusters and then send new EDS requests dropping those resources.

I am not sure the impact of this, but it seems a bit suspicious at the very least

Repro steps: Send SIGTERM to envoy, turn on trace:config logs and see the discovery requests. Note in my case it was pretty rare to actually see these show up in the XDS server, likely due to racing to shutdown the process, but it does happen. Seems like usually only a couple requests get through for some reason.

Logs: logs.txt

SIGTERM sent at 07:59:25, you can see the EDS requests at that point

Contributor guide

Tech stack
cpprest api
Domain
backendinfrastructure
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
C++Envoy internalsXDS protocol
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Investigate the Envoy shutdown sequence in source/server/server.cc and the EDS subscription logic in source/common/config/grpc subscription impl.cc. Look at the attached logs to correlate SIGTERM timing with EDS requests. The issue is that after SIGTERM, clusters are torn down and new EDS requests are sent, which may be unnecessary. Consider how to suppress EDS requests during graceful shutdown or ensure they are not sent after a certain point. No linked PRs or assignees; the issue is open and stale.
EDS requests sent after receiving SIGTERM · envoyproxy/envoy#13900 | Good First Issue