Anybody runs the HIBP password DB locally? (ideally with this latest treasure trove) I saw some converted it to a Bloom filter (which makes lots of sense: for all the passwords on which it answers 'definitely not in set', you know there's no false positive and in case it'd answer 'potentially in set' you could still query manually against the online DB).
I'll search online but if a fellow HNer runs it offline, I'm all ears...
P.S: I've got Gbit/s FTTH as well as servers in datacenters so downloading tens of gigabytes ain't an issue
I wrote a thing for this back when you could download the whole hash database as a single torrent, but I haven’t checked it since they moved over to the PwnedPasswordsDownloader system. This doesn’t use any probabilistic data structures though, it just packs the database into the smallest binary file I could come up with.
It downloads and continually updates from the upstream database while serving the identical API. On a fast link it can download the entire thing in a few hours.
It just uses a giant BoltDB file to store compressed chunks.
I'll search online but if a fellow HNer runs it offline, I'm all ears...
P.S: I've got Gbit/s FTTH as well as servers in datacenters so downloading tens of gigabytes ain't an issue