dnsmasq on the infra node forwards queries to itself: resolv.conf lists admin_ip first and dnsmasq.conf has no no-resolv

オープン
#790 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
55/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
linux, ubuntu

調査の方向性

Start with roles/node/tasks/dns.yml and roles/infra/templates/dns/dnsmasq.conf, then compare the generated resolv.conf and dnsmasq configuration on an infra node. Define the upstream-variable behavior described in the issue and verify that dnsmasq uses only the configured upstreams, internal names still resolve, and parallel uncached queries remain reliable.

索引モデルが issue の本文から書いたものです。

説明

Summary

On the infra node, dnsmasq uses itself as an upstream server. node_dns_method: add writes admin_ip at the top of /etc/resolv.conf, and the generated dnsmasq.conf has no no-resolv, so dnsmasq reads that file and forwards to its own address. On Ubuntu, systemd-resolved picks up the same file and forwards back to dnsmasq, which closes a second loop.

A single query gets through. A burst of queries does not: every apt update on the node leaves dnsmasq unable to answer for about a minute, internal *.pigsty names included.

Environment
Pigsty v4.5.0 (template unchanged on main)
OS Ubuntu 24.04 LTS
dnsmasq 2.91
systemd 255 (systemd-resolved active)
Node DNS defaults: node_dns_method: add, node_dns_servers: ['${admin_ip}']
Root cause

roles/node/tasks/dns.yml renders, on the infra node itself:

options single-request-reopen timeout:1
nameserver 10.0.0.1        # admin_ip, i.e. this node's dnsmasq
nameserver 127.0.0.53      # systemd-resolved stub

roles/infra/templates/dns/dnsmasq.conf keeps no-resolv and server= commented out, so dnsmasq takes its upstreams from that file:

dnsmasq[846]: reading /etc/resolv.conf
dnsmasq[846]: using nameserver 10.0.0.1#53
dnsmasq[846]: using nameserver 127.0.0.53#53

dnsmasq normally ignores a nameserver that matches a local address. Here it does not, because the listen address sits on an interface that comes up after dnsmasq starts (the bind-dynamic case from #761).

Because /etc/resolv.conf is now a regular file and no longer the stub symlink, systemd-resolved reads it as foreign configuration and uses 10.0.0.1 as its global DNS server (resolvectl dns shows Global: 10.0.0.1). The result is two loops: dnsmasq to dnsmasq, and dnsmasq to resolved to dnsmasq.

Symptoms

Tracing one uncached query on port 53 shows it bouncing between 127.0.0.1, the node address and the stub before it leaves the host. It still resolves in about 13 ms, which is why nothing looks broken.

Under a burst it breaks. Every morning at the apt-daily run and at any other apt update, systemd-resolved logs:

systemd-resolved: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 10.0.0.1.

A probe querying dnsmasq directly (dig @127.0.0.1) gets no answer for about a minute, for internal names as well as external ones. It recovers on its own. dnsmasq never restarts, so no failed unit shows up.

Workaround we run

The template already declares conf-dir=/etc/dnsmasq.d, and no Pigsty role cleans that directory, so we add a file there instead of patching the template:

# /etc/dnsmasq.d/10-upstream.conf
no-resolv
server=<upstream 1>
server=<upstream 2>

server= alone is not enough. The dnsmasq man page says it "does not suppress reading of /etc/resolv.conf": that takes no-resolv. Pointing resolv-file= at /run/systemd/resolve/resolv.conf does not help either, since resolved lists the node address there too.

After a restart, dnsmasq logs only the two upstreams. Internal names answer on both listen addresses. apt-get update with a parallel dig loop loses no answer. A burst of 300 uncached names, 50 at a time, gets 300 answers, and an internal name still resolves in the middle of it. systemd-resolved no longer logs "degraded feature set".

Suggestion

An infra variable for dnsmasq upstreams would fix this without touching /etc/resolv.conf. When set, the template would render no-resolv and one server= per entry. For example, dns_upstreams: [] would keep today's behaviour, and a list such as dns_upstreams: [1.1.1.1, 8.8.8.8] would switch to explicit forwarding.

Another option: skip admin_ip in node_dns_servers when the node is itself an infra node. That removes the self reference but keeps the resolved loop, so the variable looks like the more complete fix.

Happy to test a patch on this setup.

主要言語
Shell
スター
5.9k
フォーク
396
平均マージ
4日 15時間
マージ済み PR(30日)
3

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

pgsty/pigsty のほかの issue

pgsty/pigsty の issue をすべて見る

似ている issue

Shell/Bash の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。