Cluster: catalog zone transfer ACL is reset when a Secondary node restarts, so Secondary zones silently expire behind NAT or on Kubernetes

Version: 15.6.2 (also seen on 15.6.0), Docker image `technitium/dns-server`, 3-node cluster (1 Primary, 2 Secondaries) on Kubernetes (OpenShift 4.22, OVN-Kubernetes)

Setup

– Cluster domain `dns.example.com`. Nodes are registered with their public IPs (shown here with documentation addresses): ns1 (Primary) `192.0.2.10`, ns2 `198.51.100.10`, ns3 `192.0.2.11`. Node URLs are `https://nsX.dns.example.com:53443/`.

– Each server runs as a pod behind a LoadBalancer IP. When one node connects to another node's public IP from inside the cluster, the connection arrives with the **pod's IP as source** (e.g. `10.129.0.254`), not the node's public IP. This is normal Kubernetes behaviour, and the same happens behind many NAT/hairpin setups.

– To make zone transfers work, I added the pod networks to the transfer ACL of the cluster catalog zone: `cluster-catalog.dns.example.com` → Options → Zone Transfer → Network ACL = `198.51.100.10, 192.0.2.11, 10.128.0.0/14, fd01::/48`.

Problem

Whenever a **Secondary** node restarts and rejoins, the Primary rewrites the catalog zone's Network ACL back to only the Secondaries' registered IPs, and my extra entries are removed. A Primary restart keeps the list. From then on every zone transfer from the Secondaries is refused. Nothing visible happens until the zone's SOA expire time (7 days here) passes. Then the Secondaries:

– stop serving every Secondary zone whose serial changed (`REFUSED`), including the cluster domain zone;

– having lost the TLSA records in that zone, fail every heartbeat with `UntrustedRoot`, about 17,000 log lines per server per day.

For us that meant public reverse DNS for two /24s failed on 2 of 3 nameservers for 8 days before anyone noticed.

Steps to reproduce

1. In a working cluster, set the catalog zone's Zone Transfer Network ACL to the node IPs plus an extra range (e.g. `10.128.0.0/14`).
2. Restart **one Secondary** only.
3. Check the catalog zone options on the Primary (`/api/zones/options/get?zone=cluster-catalog.<domain>`).

Expected: the extra ACL entries are kept, or there is a supported way to add extra allowed addresses for cluster transfers.

Actual: the ACL is replaced with only the Secondaries' registered IPs about 1 second after the Secondary rejoins. Timeline from my test (UTC):

\“`

23:53:09 ACL = 198.51.100.10, 192.0.2.11, 10.128.0.0/14, fd01::/48

23:53:10 deleted the ns3 pod (ns1 and ns2 untouched)

23:53:26 ns3 ready → ACL = 198.51.100.10, 192.0.2.11

23:53:28 [Primary] DNS Server Cluster config file was saved: /etc/dns/cluster.config

23:53:28 [Primary] Saved zone file for domain: cluster-catalog.dns.example.com

23:53:33 [Primary] [10.129.0.254:52122] [TCP] DNS Server refused a zone transfer request since the request IP address is not allowed by the zone: dns.example.com

\“`

Secondary log once the zones expired:

\“`

DNS Server received a zone transfer response (RCODE=Refused) for 'dns.example.com' Secondary zone from: ns1.dns.example.com (192.0.2.10)

Heartbeat failed for Primary node 'ns1.dns.example.com (192.0.2.10)'.

System.Net.Http.HttpRequestException: The remote certificate is invalid because of errors in the certificate chain: UntrustedRoot (ns1.dns.example.com:53443)

“`

Suggestions (any one would solve it):

1. Keep user-added entries in the catalog zone's transfer ACL when the cluster rebuilds it, or add a separate "additional allowed networks for cluster transfers" setting.
2. Authorise cluster zone transfers by the cluster's TSIG key alone. The catalog already has a TSIG key configured, so the source IP check is redundant for cluster peers.
3. Allow node addresses to include CIDR ranges.
4. Log a warning, or show the node as degraded in the Cluster UI, when Secondary zones are failing to transfer or have expired. Right now it fails silently for a week.