When I last tried to implement this, by far the toughest part was making sure the file that’s been newly detected is done being written to. On ntfs I couldn’t find a good technique, even last modified time was not reliable. I had to watch it for changes myself.
I've done this by watching the NTFS journal which is surprisingly efficient. First I scanned the whole journal for filesystem metadata and dumped it into a SQLite database (which took about a minute), then kept it up to date which took virtually no resources. This was an absurdly faster way to search by file name, a search across the whole FS came back in milliseconds instead of Explorer's multiple minutes.