IPv6-only devices (pretty much Android) to IPv4 on internal network are coming from my public IP

So I'm pretty sure I've figured out the specific scope to this.

I have an internal server where I host media and several other things. All dual stack.

I have DHCP option 108 set on my pfsense router.

My phone and Android TV because of this, don't request IPv4 addresses. It works great for the most part and is almost always v6, but I noticed a strange issue. Occasionally for whatever reason when connecting to the internal server, the device will choose IPv4. When this happens, the server sees this device as coming from my public IP address, and therefore blocks it (per my nginx rule).

I imagine this has something to do with DNS64 or NAT64, but I'm not sure how to investigate further.

If I navigate using the IPv4 address to the port without DNS, it works fine. never mind it's still the public address, I'm just bypassing nginx [facepalm]

What might be happening here? If it matters, my DNS server is Technitium.