Repository metrics
- Stars
- (27,997 stars)
- PR merge metrics
- (Avg merge 8d) (378 merged PRs in 30d)
Description
Title: Crash on retry when writing read-only data from upstream filter.
Description: Note: this has already been run by the envoy security team, and it's been decided that we can do this in the open.
Adding read-only data to downstream filter state in an upstream filter will crash Envoy when a retry happens. For example, the following test will crash envoy:
TEST_P(MultiplexedIntegrationTest, RetryWithFilterState) {
test::integration::filters::HeaderToFilterStateFilterConfig config;
config.set_header_name("x-uh-oh-header");
config.set_state_name("uh-oh-state");
config.set_read_only(true);
envoy::extensions::filters::network::http_connection_manager::v3::HttpFilter filter;
filter.set_name("header-to-filter-state");
filter.mutable_typed_config()->PackFrom(config);
config_helper_.prependFilter(MessageUtil::getYamlStringFromMessage(filter), false);
// also add the x-uh-oh-header header.
testRetry();
}
You might think that choosing to add the header in upstream filter state (downstream_callbacks_->streamInfo().upstreamInfo()->upstreamStreamInfo()) may be a better option, but it is not. This is because the upstream stream info is reset halfway through UpstreamRequest::onPoolReady(), which may be called after the filter chain has already decoded the headers.
Call Stack:
Unhandled exception:
@ 0x562c24625bb2 Envoy::StackTrace::terminateHandler()
@ 0x562c25539886 std::__terminate()
@ 0x562c25538f7b __cxxabiv1::failed_throw()
@ 0x562c25538ef4 __cxa_throw
@ 0x562c2475f71f Envoy::StreamInfo::FilterStateImpl::setData()
@ 0x562c22d0d0e0 Envoy::HeaderToFilterStateFilter::decodeHeaders()
@ 0x562c24273aa5 Envoy::Http::FilterManager::decodeHeaders()
@ 0x562c2424869d Envoy::Router::Filter::doRetry()
@ 0x562c24bee4f7 event_process_active_single_queue
@ 0x562c24beab4c event_base_loop
@ 0x562c22433721 Envoy::Server::WorkerImpl::threadRoutine()
@ 0x562c24bdd163 Envoy::Thread::ThreadImplPosix::ThreadImplPosix()::{lambda()#1}::__invoke()
@ 0x7f65400717d9 start_thread
@ 0x7f653ffe469f clone
libc++abi: terminating due to uncaught exception of type Envoy::EnvoyException: FilterState::setData<T> called twice on same ReadOnly state.