kubernetes/ingress-nginx
GitHub で見るserver-alias pcre regex uses the ingress-nginx default tls certificate but redirect to the correct backend which has a valid * certificate that match
Open
#10,953 opened on 2024年2月1日
help wantedkind/supportneeds-priorityneeds-triagetriage/needs-information
説明
What happened: Using regex in server alias of an ingress, but ingress-nginx uses it's default tls certificate instead of the one of the ingress.
What you expected to happen: I expect ingress-nginx to use the available valid certificate for the regex I defined in the server-alias annotations and not it's default certificate.
What do you think went wrong?: There must be some kind of cache in the TLS certificate inclusion/matching in ingress-nginx that is updated using the defined ingress rules but not using the server-alias annotation. Because when I simply use a regex in server-alias it does not work. But if I add an host with the same *.subdomain.domain.com as the certificate in the ingress (and matching the regex in server-alias), and then remove this host. That server alias uses the correct tls certificate until ingress-nginx restart. What is really strange to me is even if ingress-nginx uses the wrong certificate, it forwards the request to the correct backend....
NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):
/nginx-ingress-controller --version
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: v1.9.6
Build: 6a73aa3b05040a97ef8213675a16142a9c95952a
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.21.6
-------------------------------------------------------------------------------
Kubernetes version (use kubectl version): v1.25.9
Environment:
-
Cloud provider or hardware configuration:
OVH cloud provider -
OS (e.g. from /etc/os-release):
Ubuntu 22.04.2 LTS -
Kernel (e.g.
uname -a):5.15.0-71-generic -
Install tools:
Cloud provider managed cluster on top of openstack -
Basic cluster related info:
kubectl versionServer Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.9", GitCommit:"a1a87a0a2bcd605820920c6b0e618a8ab7d117d4", GitTreeState:"clean", BuildDate:"2023-04-12T12:08:36Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/amd64"}kubectl get nodes -o wideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME REDACTED Ready <none> 273d v1.25.9 10.1.0.198 <none> Ubuntu 22.04.2 LTS 5.15.0-71-generic containerd://1.6.18 REDACTED Ready <none> 273d v1.25.9 10.1.2.160 <none> Ubuntu 22.04.2 LTS 5.15.0-71-generic containerd://1.6.18 REDACTED Ready <none> 273d v1.25.9 10.1.3.228 <none> Ubuntu 22.04.2 LTS 5.15.0-82-generic containerd://1.6.18
-
How was the ingress-nginx-controller installed:
- If helm was used then please show output of
helm ls -A | grep -i ingressNAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION ingress-nginx ingress-nginx xx xx deployed ingress-nginx-4.9.1 1.9.6- If helm was used then please show output of
helm -n <ingresscontrollernamespace> get values <helmreleasename>
controller: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app.kubernetes.io/component: controller app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx namespaces: - ingress-nginx topologyKey: kubernetes.io/hostname allowSnippetAnnotations: true extraArgs: default-ssl-certificate: ingress-nginx/default-tls-secret extraVolumeMounts: - mountPath: /etc/nginx/modsecurity/custom name: custom-modsecurity-configmap extraVolumes: - configMap: name: custom-modsecurity-configmap name: custom-modsecurity-configmap kind: Deployment replicaCount: 2 service: enableHttp: false - If helm was used then please show output of
- If helm was used then please show output of
-
Current State of the controller:
-
kubectl describe ingressclassesName: nginx Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.9.6 helm.sh/chart=ingress-nginx-4.9.1 Annotations: meta.helm.sh/release-name: ingress-nginx meta.helm.sh/release-namespace: ingress-nginx Controller: k8s.io/ingress-nginx Events: <none> -
kubectl -n <ingresscontrollernamespace> get all -A -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/fluentbit-waf-fluent-bit-b67cc69c7-bvmt6 1/1 Running 0 157d 10.2.1.193 REDACTED <none> <none> pod/ingress-nginx-controller-6cf8c5fd8f-6j7qr 1/1 Running 0 5d23h 10.2.1.90 REDACTED <none> <none> pod/ingress-nginx-controller-6cf8c5fd8f-x9nqf 1/1 Running 0 5d23h 10.2.2.159 REDACTED <none> <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/fluentbit-waf-fluent-bit ClusterIP 10.3.156.145 <none> 2020/TCP,8888/TCP 250d app.kubernetes.io/instance=fluentbit-waf,app.kubernetes.io/name=fluent-bit service/ingress-nginx-controller LoadBalancer 10.3.156.241 REDACTED 443:31257/TCP 273d app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx service/ingress-nginx-controller-admission ClusterIP 10.3.188.238 <none> 443/TCP 273d app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR deployment.apps/fluentbit-waf-fluent-bit 1/1 1 1 250d fluent-bit fluent/fluent-bit:1.8.11 app.kubernetes.io/instance=fluentbit-waf,app.kubernetes.io/name=fluent-bit deployment.apps/ingress-nginx-controller 2/2 2 2 5d23h controller registry.k8s.io/ingress-nginx/controller:v1.9.6@sha256:1405cc613bd95b2c6edd8b2a152510ae91c7e62aea4698500d23b2145960ab9c app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR replicaset.apps/fluentbit-waf-fluent-bit-b67cc69c7 1 1 1 239d fluent-bit fluent/fluent-bit:1.8.11 app.kubernetes.io/instance=fluentbit-waf,app.kubernetes.io/name=fluent-bit,pod-template-hash=b67cc69c7 replicaset.apps/ingress-nginx-controller-6cf8c5fd8f 2 2 2 5d23h controller registry.k8s.io/ingress-nginx/controller:v1.9.6@sha256:1405cc613bd95b2c6edd8b2a152510ae91c7e62aea4698500d23b2145960ab9c app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx,pod-template-hash=6cf8c5fd8f -
kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>Name: ingress-nginx-controller-6cf8c5fd8f-6j7qr Namespace: ingress-nginx Priority: 0 Service Account: ingress-nginx Node: REDACTED/10.1.2.160 Start Time: Wed, 31 Jan 2024 18:18:34 +0100 Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.9.6 helm.sh/chart=ingress-nginx-4.9.1 pod-template-hash=6cf8c5fd8f Annotations: cni.projectcalico.org/containerID: 4c36b01a35fa9ce3c783edf7794969d853d8a30a09938c1b0f84b299c98cc8e7 cni.projectcalico.org/podIP: 10.2.1.90/32 cni.projectcalico.org/podIPs: 10.2.1.90/32 Status: Running IP: 10.2.1.90 IPs: IP: 10.2.1.90 Controlled By: ReplicaSet/ingress-nginx-controller-6cf8c5fd8f Containers: controller: Container ID: containerd://be000566c8ecf666f3ddb9d70dd44d1271630011f3e483094e1f7883ecc9f4c7 Image: registry.k8s.io/ingress-nginx/controller:v1.9.6@sha256:1405cc613bd95b2c6edd8b2a152510ae91c7e62aea4698500d23b2145960ab9c Image ID: registry.k8s.io/ingress-nginx/controller@sha256:1405cc613bd95b2c6edd8b2a152510ae91c7e62aea4698500d23b2145960ab9c Ports: 80/TCP, 443/TCP, 8443/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP Args: /nginx-ingress-controller --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller --election-id=ingress-nginx-leader --controller-class=k8s.io/ingress-nginx --ingress-class=nginx --configmap=$(POD_NAMESPACE)/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key --default-ssl-certificate=ingress-nginx/k8s-default-tls-secret State: Running Started: Wed, 31 Jan 2024 18:18:36 +0100 Ready: True Restart Count: 0 Requests: cpu: 100m memory: 90Mi Liveness: http-get http://:10254/healthz delay=10s timeout=15s period=10s #success=1 #failure=5 Readiness: http-get http://:10254/healthz delay=10s timeout=15s period=10s #success=1 #failure=3 Environment: POD_NAME: ingress-nginx-controller-6cf8c5fd8f-6j7qr (v1:metadata.name) POD_NAMESPACE: ingress-nginx (v1:metadata.namespace) LD_PRELOAD: /usr/local/lib/libmimalloc.so Mounts: /etc/nginx/modsecurity/custom from custom-modsecurity-configmap (rw) /usr/local/certificates/ from webhook-cert (ro) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cddh8 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: webhook-cert: Type: Secret (a volume populated by a Secret) SecretName: ingress-nginx-admission Optional: false custom-modsecurity-configmap: Type: ConfigMap (a volume populated by a ConfigMap) Name: custom-modsecurity-configmap Optional: false kube-api-access-cddh8: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: Burstable Node-Selectors: kubernetes.io/os=linux Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: <none> -
kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>Name: ingress-nginx-controller Namespace: ingress-nginx Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.9.6 helm.sh/chart=ingress-nginx-4.9.1 Annotations: meta.helm.sh/release-name: ingress-nginx meta.helm.sh/release-namespace: ingress-nginx service.beta.kubernetes.io/ovh-loadbalancer-proxy-protocol: v2 Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx Type: LoadBalancer IP Family Policy: SingleStack IP Families: IPv4 IP: 10.3.156.241 IPs: 10.3.156.241 LoadBalancer Ingress: REDACTED Port: https 443/TCP TargetPort: https/TCP NodePort: https 31257/TCP Endpoints: 10.2.1.90:443,10.2.2.159:443 Session Affinity: None External Traffic Policy: Cluster Events: <none>
-
-
Current state of ingress object, if applicable:
-
kubectl -n <appnamespace> get all,ing -o wideNAME CLASS HOSTS ADDRESS PORTS AGE foo nginx subdomain.domain.com REDACTED 80, 443 6d23h bar nginx test.subdomain.domain.com REDACTED 80, 443 6d23h -
kubectl -n <appnamespace> describe ing <ingressname>Name: foo Labels: <none> Namespace: test1 Address: REDACTED Ingress Class: nginx Default backend: <default> TLS: k8s-tls-secret terminates *.subdomain.domain.com,subdomain.domain.com Rules: Host Path Backends ---- ---- -------- subdomain.domain.com echo1:80 (10.2.2.148:5678) Annotations: cert-manager.io/cluster-issuer: letsencrypt-prod nginx.ingress.kubernetes.io/server-alias: ~^(?!test).+\.subdomain\.domain\.com$' Events: <none> Name: bar Labels: <none> Namespace: test1 Address: REDACTED Ingress Class: nginx Default backend: <default> TLS: k8s-tls-secret terminates Rules: Host Path Backends ---- ---- -------- test.subdomain.domain.com echo2:80 (10.2.2.147:5678) Annotations: nginx.ingress.kubernetes.io/configuration-snippet: set $svr_ssl_client_cert ''; access_by_lua_block { ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or "" } proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert; nginx.ingress.kubernetes.io/server-alias: ~^test-(.+)\.subdomain\.domain\.com$' nginx.ingress.kubernetes.io/server-snippet: ssl_verify_client optional_no_ca; Events: <none> -
If applicable, then, your complete and exact curl/grpcurl command (redacted if required) and the reponse to the curl/grpcurl command with the -v flag
-
-
Others:
- Any other related information like ;
-
copy/paste of the snippet (if applicable)
-
kubectl describe ...of any custom configmap(s) created and in useName: ingress-nginx-controller Namespace: ingress-nginx Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.9.6 helm.sh/chart=ingress-nginx-4.9.1 Annotations: meta.helm.sh/release-name: ingress-nginx meta.helm.sh/release-namespace: ingress-nginx Data ==== proxy-read-timeout: ---- 300 ssl-session-cache-size: ---- 50m allow-snippet-annotations: ---- true hsts-include-subdomains: ---- true ssl-session-cache: ---- true ssl-session-tickets: ---- false ssl-session-timeout: ---- 1d proxy-connect-timeout: ---- 300 proxy-headers-hash-max-size: ---- 1024 proxy-send-timeout: ---- 300 ssl-protocols: ---- TLSv1.2 TLSv1.3 hsts: ---- true hsts-max-age: ---- 31536000 http-snippet: ---- modsecurity_transaction_id "$request_id"; server-snippet: ---- # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; use-gzip: ---- true location-snippet: ---- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; modsecurity-snippet: ---- ## Reference manual github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v3.x) # Enable prevention mode. Can be any of: DetectionOnly,On,Off (default is DetectionOnly) SecRuleEngine DetectionOnly # Max request sizes in bytes (with/without files) # Note NGINX Ingress has its own annotations, keep in sync! SecRequestBodyLimit 1048576000 # 1000Mb SecRequestBodyNoFilesLimit 1048576000 # 1000Mb SecRequestBodyLimitAction ProcessPartial # Can be set to "Reject" if larger (we let it pass with "ProcessPartial" for now) # SecDebugLog /tmp/modsec_debug.log SecAuditEngine Off SecAuditLogFormat JSON SecAuditLogType HTTPS SecAuditLog "http://fluentbit-waf-fluent-bit.ingress-nginx.svc.cluster.local:8888" # Allow ingress-nginx to check its own status using localhost ip 127.0.0.1 SecRule REMOTE_ADDR "@ipMatch 127.0.0.1" "id:87,phase:1,pass,nolog,ctl:ruleEngine=Off" # Set paranoia level 1 (default out of 1->3) # https://coreruleset.org/docs/concepts/paranoia_levels/ SecAction "id:900000,phase:1,nolog,pass,t:none,setvar:tx.paranoia_level=1" # Rule to allow http methods GET POST PUT DELETE HEAD OPTIONS ## If required add PATCH SecAction "id:900200,phase:1,nolog,pass,t:none,setvar:tx.allowed_methods=GET POST PUT DELETE HEAD OPTIONS" ## TODO ## We can setup the WAF in blocking mode and increase the blocking threshold to ## handle less False Positive at first and progressively lower the threshold to the defaults (5 inbound 4 outbound) SecAction \ "id:900110,\ phase:1,\ nolog,\ pass,\ t:none,\ setvar:tx.inbound_anomaly_score_threshold=10000,\ setvar:tx.outbound_anomaly_score_threshold=10000" # Handle Body size > SecRequestBodyLimit when using SecRequestBodyLimitAction ProcessPartial SecRule INBOUND_DATA_ERROR "@eq 1" "phase:2,id:24,t:none,log,pass,msg:\'Request Body Larger than SecRequestBodyLimit Setting\'" Include /etc/nginx/modsecurity/custom/modsecurity.conf Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf proxy-headers-hash-bucket-size: ---- 128 real-ip-header: ---- proxy_protocol server-names-hash-bucket-size: ---- 128 use-proxy-protocol: ---- true use-http2: ---- false enable-modsecurity: ---- true hsts-preload: ---- true server-name-hash-max-size: ---- 1024 ssl-ciphers: ---- EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH BinaryData ==== Events: <none> -
Any other related information that may help
## start server subdomain.domain.com server { server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "test1"; set $ingress_name "foo"; set $service_name "echo1"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo1-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server subdomain.domain.com ## start server test.subdomain.domain.com server { server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } # Custom code snippet configured for host test.subdomain.domain.com ssl_verify_client optional_no_ca; location / { set $namespace "test1"; set $ingress_name "bar"; set $service_name "echo2"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo2-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; set $svr_ssl_client_cert ''; access_by_lua_block { ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or "" } proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server test.subdomain.domain.com
-
- Any other related information like ;
How to reproduce this issue:
Install an application that will act as default backend (is just an echo app)
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/http-svc.yaml
Create an ingress (please add any additional annotation required)
Need tls certificate for this ingress (example using cert-manager)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: foo
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/server-alias: '~^(?!test).+\.subdomain\.domain\.com$'
spec:
tls:
- hosts:
- '*.subdomain.domain.com'
- 'subdomain.domain.com'
secretName: tls-secret
rules:
- host: 'subdomain.domain.com'
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: http-svc
port:
number: 80
ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bar
annotations:
nginx.ingress.kubernetes.io/server-alias: '~^test-(.+)\.subdomain\.domain\.com$'
spec:
tls:
- hosts:
- '*.subdomain.domain.com'
- 'subdomain.domain.com'
secretName: tls-secret
rules:
- host: 'test.subdomain.domain.com'
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: http-svc
port:
number: 80
ingressClassName: nginx
## start server subdomain.domain.com
server {
server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "test1";
set $ingress_name "foo";
set $service_name "echo1";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo1-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server subdomain.domain.com
## start server test.subdomain.domain.com
server {
server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
# Custom code snippet configured for host test.subdomain.domain.com
ssl_verify_client optional_no_ca;
location / {
set $namespace "test1";
set $ingress_name "bar";
set $service_name "echo2";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo2-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
set $svr_ssl_client_cert '';
access_by_lua_block {
ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or ""
}
proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server test.subdomain.domain.com
make a request
# Uses correct certificate
curl -v subdomain.domain.com
curl -v test.subdomain.domain.com
# Uses wrong certificate
curl -v random.subdomain.domain.com
curl -v test-random.subdomain.domain.com
Anything else we need to know: if you apply the following ingress definition, and then re-apply the first one. The correct certficate match with the server-alias regex until ingress-nginx restart and start using it's default certificate all over again...
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: foo
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/server-alias: '~^(?!test).+\.subdomain\.domain\.com$'
spec:
tls:
- hosts:
- '*.subdomain.domain.com'
- 'subdomain.domain.com'
secretName: tls-secret
rules:
- host: 'subdomain.domain.com'
http: &paths
paths:
- pathType: ImplementationSpecific
backend:
service:
name: http-svc
port:
number: 80
- host: '*.subdomain.domain.com'
http: *paths
ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bar
annotations:
nginx.ingress.kubernetes.io/server-alias: '~^test-(.+)\.subdomain\.domain\.com$'
spec:
tls:
- hosts:
- '*.subdomain.domain.com'
- 'subdomain.domain.com'
secretName: tls-secret
rules:
- host: 'test.subdomain.domain.com'
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: http-svc
port:
number: 80
ingressClassName: nginx
## start server *.subdomain.domain.com
server {
server_name ~^(?<subdomain>[\w-]+)\.subdomain\.domain\.com$ ~^(?!test).+\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "test1";
set $ingress_name "foo";
set $service_name "echo1";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo1-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server *.subdomain.domain.com
## start server subdomain.domain.com
server {
server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "test1";
set $ingress_name "foo";
set $service_name "echo1";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo1-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server subdomain.domain.com
## start server test.subdomain.domain.com
server {
server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
# Custom code snippet configured for host test.subdomain.domain.com
ssl_verify_client optional_no_ca;
location / {
set $namespace "test1";
set $ingress_name "bar";
set $service_name "echo2";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo2-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
set $svr_ssl_client_cert '';
access_by_lua_block {
ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or ""
}
proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server test.subdomain.domain.com
## start server subdomain.domain.com
server {
server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "test1";
set $ingress_name "foo";
set $service_name "echo1";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo1-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server subdomain.domain.com
## start server test.subdomain.domain.com
server {
server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ;
listen 80 proxy_protocol ;
listen [::]:80 proxy_protocol ;
listen 443 proxy_protocol ssl ;
listen [::]:443 proxy_protocol ssl ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
# Custom code snippet configured for host test.subdomain.domain.com
ssl_verify_client optional_no_ca;
location / {
set $namespace "test1";
set $ingress_name "bar";
set $service_name "echo2";
set $service_port "80";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "test1-echo2-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $proxy_protocol_server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
set $svr_ssl_client_cert '';
access_by_lua_block {
ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or ""
}
proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert;
# Custom code snippet configured in the configuration configmap
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# Custom code snippet configured in the configuration configmap
# Prevent clickjacking attacks
# Comment out this line if you want to make use of iframes
add_header X-Frame-Options SAMEORIGIN;
# Prevent "mime" based attacks
add_header X-Content-Type-Options nosniff;
# Prevent XSS attacks
add_header X-XSS-Protection "1; mode=block";
}
## end server test.subdomain.domain.com