Hi all,

I found a bit of a peculiar issue that is affecting Tailscale Magic DNS on-device proxy (100.100.100.100) on my iPhone. I didn't spot this issue on my Mac, though I can't confirm it's not there, but it's maybe just less visible.

I'm wondering if any one is / was facing the same problem and if it would be worth reporting to ControlD Support.

Setup

Small tailnet of six devices. Magic DNS is enabled and Global server is pointing to a specific DoH ControlD Endpoint. I also have two domains that I resolve via an internal server.

MAGIC_DNS_PROXY -> CONTROLD_DOH \> INTERNAL_DNS 

The issue

A couple weeks ago, I noticed that internet connections were breaking randomly when connected to Tailscale, and I had to restart the VPN manually when this happened. I was able to pinpoint that down to the Magic DNS. When this happened, I couldn't even ping 100.100.100.100.

I went down the rabbit hole a bit. First, I suspected that this might be related to the Magic DNS process not handling DoH well, so I deployed an internal DNS proxy that's reachable via the tailnet and it's relaying to the DoH Endpoint. Unfortunately, that did not help.

The confusing part is that I didn't really spot any network and / or resolving errors at first sight.

When the DNS went bananas, I could see this error message in the Tailscale app:

DNS Unavailable Tailscale can't reach the configured DNS servers. Internet connectivity may be affected. Code: dns-forward-failing 

And these logs repeating over and over:

... [general] - gui: [ipnBus] - Stopping IPN Watch Session ... [general] - gui: [ipnBus] - Started IPN Watch Session ... LocalAPI request to <none> failed with localAPIStatusError(status: 500...) IPNBusMessageConsumer error: NSURLErrorDomain Code=-1001 "The request timed out" ... 

At this point, I had to restart the VPN.

I started reviewing DNS requests and responses (even those that were legitimately blocked) on my laptop and also the internal DNS proxy. That's when I spotted that ControlD responded with REFUSED to HTTPS/SVCB queries.

$ dig +https=/<resolver-id>/digtest @dns.controld.com api3.siftscience.com TYPE65 ;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 34350 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;api3.siftscience.com. IN HTTPS ;; AUTHORITY SECTION: . 10 IN SOA dns.controld.com. hostmaster.api3.siftscience.com. 202300 10 10 10 10 ;; SERVER: 76.76.2.22#443(dns.controld.com) (HTTPS) 

I kinda remembered that this might not be correct, so I started reading the RFC9460.

• Section 4.2 is basically warning about the REFUSED responses.

Errors or anomalies in obtaining additional records MAY cause this process to terminate but MUST NOT themselves cause the resolver to send a failure response.

• Sections 5.1 and 10.4.4 treat NODATA as legitimate and something that clients will handle by falling back to A/AAAA.

The workaround

After discovering the REFUSED responses, I decided to switch the ControlD profile to NXDOMAIN instead of 'Branded'.

$ dig +https=/<resolver-id>/digtest @dns.controld.com api3.siftscience.com TYPE65 ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 11469 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;api3.siftscience.com. IN HTTPS ;; AUTHORITY SECTION: . 10 IN SOA dns.controld.com. hostmaster.api3.siftscience.com. 202300 10 10 10 10 ;; SERVER: 76.76.2.22#443(dns.controld.com) (HTTPS) 

That immediately fixed the (Magic) DNS issues and the VPN is stable.

Fin

I don't think that the 'Branded' blocking option should respond with REFUSED to HTTPS/SVCB queries, but instead rather default to NODATA.

Thoughts on this?