Built this for my own setup. `sieve devices` finds ESPHome, WLED and Victron Venus boards over mDNS and checks:
– ESPHome: firmware version vs known advisories, API encryption, OTA password, whether the web UI has a login
– WLED: version + whether OTA is locked
– Venus OS: Remote Console login, Modbus TCP, plain MQTT on 1883
It doesn't upload anything or try logins – the OTA check stops at the point the board asks for a password.
On mine it flagged a WLED strip with OTA unlocked and an ESPHome board with no web login. Both fixed in 5 minutes.
pipx install sieve-scan, then: sieve devices
https://github.com/casareanderson/sieve
Free, AGPL. Only tested on my own kit, so I'd love to hear what it gets wrong on yours.
Source: r/homeassistant · by /u/casareanderson