envoyproxy/envoy

Assertion failure when RBAC filter config is invalid

Open

#5,324 opened on 2018年12月17日

GitHub で見る
 (10 comments) (0 reactions) (0 assignees)C++ (5,373 forks)batch import
bughelp wanted

Repository metrics

Stars
 (27,997 stars)
PR merge metrics
 (平均マージ 8d) (30d で 378 merged PRs)

説明

Title: Assertion failure when RBAC filter config is invalid

Description: Envoy should ignore / gracefully warn/fail configuration errors in the filter config for a listener. Instead, some assertions are being called on filter creation.

Repro steps: Using Istio with docker.io/istio/proxyv2:1.0.3 of the container.

Add this config:

---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingress-ip-restrictions
  namespace: istio-system
spec:
  workloadLabels:
    istio: ingressgateway
  filters:
    - filterName: envoy.filters.network.rbac
      filterType: NETWORK
      filterConfig:
        rules:
          action: ALLOW
          policies:
            "ip-restrictions":
              permissions:
                - any: true
              principals:
                - source_ip:
                  address_prefix: 1.2.3.4
                  prefix_len: 32
                - source_ip:
                  address_prefix: 5.6.7.8
                  prefix_len: 32

        stat_prefix: ingress-ip-restrictions

(Note that the indentation is incorrect so the source_ip won't be a valid CIDR. Also saw similar crashes when principals was a hash not an array, etc.)

Call Stack:

[2018-12-17 04:16:52.963][47][info][main] external/envoy/source/server/server.cc:401] all clusters initialized. initializing init manager
[2018-12-17 04:16:52.968][47][critical][assert] external/envoy/source/extensions/filters/common/rbac/matchers.cc:53] panic: not reached
[2018-12-17 04:16:52.968][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Aborted, suspect faulting address 0x2f
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<0> obj</lib/x86_64-linux-gnu/libc.so.6> (If unsymbolized, use tools/stack_decode.py):
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #0 0x7f70860ca428 (unknown)
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #1 0x7f70860cc029 (unknown)
[2018-12-17 04:16:52.969][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</usr/local/bin/envoy>
[2018-12-17 04:16:52.973][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #2 0x6a7876 Envoy::Extensions::Filters::Common::RBAC::Matcher::create()
[2018-12-17 04:16:52.976][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #3 0x6a7a5c Envoy::Extensions::Filters::Common::RBAC::OrMatcher::OrMatcher()
[2018-12-17 04:16:52.980][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #4 0x6a5548 Envoy::Extensions::Filters::Common::RBAC::RoleBasedAccessControlEngineImpl::RoleBasedAccessControlEngineImpl()
[2018-12-17 04:16:52.984][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #5 0x6a3d25 Envoy::Extensions::NetworkFilters::RBACFilter::RoleBasedAccessControlFilterConfig::RoleBasedAccessControlFilterConfig()
[2018-12-17 04:16:52.987][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #6 0x6a3586 Envoy::Extensions::NetworkFilters::RBACFilter::RoleBasedAccessControlNetworkFilterConfigFactory::createFilterFactoryFromProtoTyped()
[2018-12-17 04:16:52.991][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #7 0x6a24e8 Envoy::Extensions::NetworkFilters::Common::FactoryBase<>::createFilterFactoryFromProto()
[2018-12-17 04:16:52.995][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #8 0x7a6363 Envoy::Server::ProdListenerComponentFactory::createNetworkFilterFactoryList_()
[2018-12-17 04:16:52.998][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #9 0x7aee76 Envoy::Server::ListenerImpl::ListenerImpl()
[2018-12-17 04:16:53.002][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #10 0x7b049b Envoy::Server::ListenerManagerImpl::addOrUpdateListener()
[2018-12-17 04:16:53.006][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #11 0x7bf6f0 Envoy::Server::LdsApiImpl::onConfigUpdate()
[2018-12-17 04:16:53.010][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #12 0x7c178f Envoy::Config::GrpcMuxSubscriptionImpl<>::onConfigUpdate()
[2018-12-17 04:16:53.013][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #13 0x8fd4bc Envoy::Config::GrpcMuxImpl::onReceiveMessage()
[2018-12-17 04:16:53.015][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #14 0x8fab82 Envoy::Grpc::TypedAsyncStreamCallbacks<>::onReceiveMessageUntyped()
[2018-12-17 04:16:53.018][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #15 0x9250a5 Envoy::Grpc::AsyncStreamImpl::onData()
[2018-12-17 04:16:53.021][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #16 0x929dcb Envoy::Http::AsyncStreamImpl::encodeData()
[2018-12-17 04:16:53.023][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #17 0x966853 Envoy::Http::Http2::ConnectionImpl::onFrameReceived()
[2018-12-17 04:16:53.026][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #18 0x96a716 nghttp2_session_on_data_received
[2018-12-17 04:16:53.029][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #19 0x96e831 nghttp2_session_mem_recv
[2018-12-17 04:16:53.031][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #20 0x96641e Envoy::Http::Http2::ConnectionImpl::dispatch()
[2018-12-17 04:16:53.034][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #21 0x918fbe Envoy::Http::CodecClient::onData()
[2018-12-17 04:16:53.037][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #22 0x91913c Envoy::Http::CodecClient::CodecReadFilter::onData()
[2018-12-17 04:16:53.039][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #23 0x7d11d6 Envoy::Network::FilterManagerImpl::onContinueReading()
[2018-12-17 04:16:53.042][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #24 0x7cf85e Envoy::Network::ConnectionImpl::onReadReady()
[2018-12-17 04:16:53.045][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #25 0x7d004d Envoy::Network::ConnectionImpl::onFileEvent()
[2018-12-17 04:16:53.047][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #26 0x7c94a7 Envoy::Event::FileEventImpl::assignEvents()::{lambda()#1}::_FUN()
[2018-12-17 04:16:53.050][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #27 0xae4354 event_process_active_single_queue.isra.29
[2018-12-17 04:16:53.053][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #28 0xae4a9e event_process_active
[2018-12-17 04:16:53.055][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #29 0xae7827 event_base_loop
[2018-12-17 04:16:53.058][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #30 0x7989a3 Envoy::Server::InstanceImpl::run()
[2018-12-17 04:16:53.061][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #31 0x578600 Envoy::MainCommonBase::run()
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #32 0x422ea2 main
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</lib/x86_64-linux-gnu/libc.so.6>
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<0> #33 0x7f70860b582f (unknown)
[2018-12-17 04:16:53.065][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<0> obj</usr/local/bin/envoy>
[2018-12-17 04:16:53.068][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<0> #34 0x47bdd8 _start
[2018-12-17 04:16:53.068][47][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:121] end backtrace thread 0
2018-12-17T04:16:53.206258Z	warn	Epoch 0 terminated with an error: signal: aborted (core dumped)
2018-12-17T04:16:53.206298Z	warn	Aborted all epochs
2018-12-17T04:16:53.206371Z	info	Epoch 0: set retry delay to 3.2s, budget to 5

/cc @lizan @grosser

コントリビューターガイド

調査方針
source/extensions/filters/common/rbac/matchers.cc:53 のアサーションを調べ、無効な CIDR 入力がどのようにそれをトリガーするかを追跡します。
技術スタック
cpp
領域
backend
Issue 種別
バグ
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
3
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
半日
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
古い
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
明確
前提条件
C++RBAC
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
60