Just reporting in some plain text format so I can manually delete the duplicates...

nikisweeting · 2024-11-03T04:08:59 1730606939

Here you go:

    cd /path/to/drive
    find . -type f -exec sha256sum {} + | sed -E 's/^([^ ]+) \./\1,/' >> ~/all_hashes.txt

Run that for each drive, then when you're done run:

    sort ~/all_hashes.txt > ~/sorted_hashes.txt
    awk -F, 'NR==1{print;next} {print $0 | "sort | uniq -w64 -D"}' ~/sorted_hashes.txt > ~/non_unique_hashes.txt

The output in ~/non_unique_hashes.txt will contain only the non-unique hashes that appear on more than one path.