Posting in case it saves someone a bad day, and to see if others have seen it.
Setup: HA Grid Master pair, anycast DNS advertised over eBGP to our upstream routers on LAN2. Worked great for a long time. We upgraded 9.0.8 → 9.1.0 and anycast
DNS started blackholing for off-subnet clients. Standalone Grid members in the same grid, same 9.1.0, same design — totally fine.
Dug in and found two things, and I think both are baked into 9.1.0, not our config:
1. The HA member can't initiate BGP. NIOS generates neighbor <peer> update-source <LAN1 VIP> on the HA member — but on an HA pair that VIP lives on the HA port,
which is a different subnet from the LAN2 peers. So outbound sessions get rejected; they only come up when the router initiates. The standalone member generates
no update-source and just works.
The kicker: in the WAPI schema the BGP neighbor interface field only accepts LAN_HA — there's literally no option to source BGP from LAN2. So there's no config
knob to fix it.
2. The BGP default never installs into the anycast route table. Even with the session up and the default marked best in show bgp, it's in no kernel table. The
anycast table has a kernel/boot default that beats eBGP on admin distance:
HA member (broken): default via <internal gw> dev eth2 <- kernel route, distance 0
standalone (works): default nhid 32 proto bgp metric 20 <- eBGP installs
So replies from the anycast address get shoved out the HA port and dropped. Classic return-path black hole — forward path is fine, named answers, replies just
die.
Regression: this is the Quagga → FRRouting swap in 9.1.0. FYI the FRR build string is 10.5-dev-MyOwnFRRVersion (a dev placeholder), and it's crash-restarting the
routing daemon on both members occasionally too.
Our fix was to just pull anycast off the HA Grid Master and serve it only from standalone members (which never had the problem). Honestly anycast already gives
you the redundancy, so stacking it on HA was arguably the wrong call anyway.
Questions for the hive mind:
– Anyone else running anycast BGP on an HA pair hit this after 9.1.0?
– Is there a known bug ID / hotfix?
– Any supported way to control the generated update-source on an HA member, or is anycast-on-HA just unsupported now?
Opening a TAC case regardless, but figured this can't be unique to us. 🙃
Source: r/Infoblox · by /u/TechETS