dnsmasq on the infra node forwards queries to itself: resolv.conf lists admin_ip first and dnsmasq.conf has no no-resolv

未关闭
  • 哪里不对？
#790 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
55/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
linux, ubuntu
领域
infrastructure, networking

调研方向

Start with roles/node/tasks/dns.yml and roles/infra/templates/dns/dnsmasq.conf, then compare the generated resolv.conf and dnsmasq configuration on an infra node. Define the upstream-variable behavior described in the issue and verify that dnsmasq uses only the configured upstreams, internal names still resolve, and parallel uncached queries remain reliable.

由索引模型根据 Issue 内容生成。

描述

Summary

On the infra node, dnsmasq uses itself as an upstream server. node_dns_method: add writes admin_ip at the top of /etc/resolv.conf, and the generated dnsmasq.conf has no no-resolv, so dnsmasq reads that file and forwards to its own address. On Ubuntu, systemd-resolved picks up the same file and forwards back to dnsmasq, which closes a second loop.

A single query gets through. A burst of queries does not: every apt update on the node leaves dnsmasq unable to answer for about a minute, internal *.pigsty names included.

Environment
Pigsty v4.5.0 (template unchanged on main)
OS Ubuntu 24.04 LTS
dnsmasq 2.91
systemd 255 (systemd-resolved active)
Node DNS defaults: node_dns_method: add, node_dns_servers: ['${admin_ip}']
Root cause

roles/node/tasks/dns.yml renders, on the infra node itself:

options single-request-reopen timeout:1
nameserver 10.0.0.1        # admin_ip, i.e. this node's dnsmasq
nameserver 127.0.0.53      # systemd-resolved stub

roles/infra/templates/dns/dnsmasq.conf keeps no-resolv and server= commented out, so dnsmasq takes its upstreams from that file:

dnsmasq[846]: reading /etc/resolv.conf
dnsmasq[846]: using nameserver 10.0.0.1#53
dnsmasq[846]: using nameserver 127.0.0.53#53

dnsmasq normally ignores a nameserver that matches a local address. Here it does not, because the listen address sits on an interface that comes up after dnsmasq starts (the bind-dynamic case from #761).

Because /etc/resolv.conf is now a regular file and no longer the stub symlink, systemd-resolved reads it as foreign configuration and uses 10.0.0.1 as its global DNS server (resolvectl dns shows Global: 10.0.0.1). The result is two loops: dnsmasq to dnsmasq, and dnsmasq to resolved to dnsmasq.

Symptoms

Tracing one uncached query on port 53 shows it bouncing between 127.0.0.1, the node address and the stub before it leaves the host. It still resolves in about 13 ms, which is why nothing looks broken.

Under a burst it breaks. Every morning at the apt-daily run and at any other apt update, systemd-resolved logs:

systemd-resolved: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 10.0.0.1.

A probe querying dnsmasq directly (dig @127.0.0.1) gets no answer for about a minute, for internal names as well as external ones. It recovers on its own. dnsmasq never restarts, so no failed unit shows up.

Workaround we run

The template already declares conf-dir=/etc/dnsmasq.d, and no Pigsty role cleans that directory, so we add a file there instead of patching the template:

# /etc/dnsmasq.d/10-upstream.conf
no-resolv
server=<upstream 1>
server=<upstream 2>

server= alone is not enough. The dnsmasq man page says it "does not suppress reading of /etc/resolv.conf": that takes no-resolv. Pointing resolv-file= at /run/systemd/resolve/resolv.conf does not help either, since resolved lists the node address there too.

After a restart, dnsmasq logs only the two upstreams. Internal names answer on both listen addresses. apt-get update with a parallel dig loop loses no answer. A burst of 300 uncached names, 50 at a time, gets 300 answers, and an internal name still resolves in the middle of it. systemd-resolved no longer logs "degraded feature set".

Suggestion

An infra variable for dnsmasq upstreams would fix this without touching /etc/resolv.conf. When set, the template would render no-resolv and one server= per entry. For example, dns_upstreams: [] would keep today's behaviour, and a list such as dns_upstreams: [1.1.1.1, 8.8.8.8] would switch to explicit forwarding.

Another option: skip admin_ip in node_dns_servers when the node is itself an infra node. That removes the self reference but keeps the resolved loop, so the variable looks like the more complete fix.

Happy to test a patch on this setup.

主要语言
Shell
星标
5.9k
派生
396
平均合并
3 天 11 小时
30 天内合并 PR
4

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

pgsty/pigsty 的其他 Issue

查看 pgsty/pigsty 的全部 Issue

相似的 Issue

更多 Shell/Bash Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。