Hacker News new | past | comments | ask | show | jobs | submit login

https://meldmerge.org/ might be what you are looking for. But if there are tools which can find duplicated files and directories which are named differently, I'd be interested in learning more about these as well.



meld can compare two directory if they have the same structure and name for files but can't really tell if I inside my foot folder, I have a folder root/X which is also present as root/category/b/X. (except all the file in it have been prepended by "foo" somehow)

For tools which can find differently names files you have for examples: fslint (gui) or fdupes (cli)


That's true, meld is designed for comparing directories which already have similar structures.

It seems like what we're looking for is content-addressable storage [1]. The theory behind it appears to be based on Merkle trees and cryptographic signatures [2].

IPFS already has an implementation [3] of this, and there are other implementations (borgbackup, restic, zpaq) listed in that link and in the content-addressable storage article. Disclaimer: I haven't used any of these yet, just found them a few minutes ago.

[1] https://en.wikipedia.org/wiki/Content-addressable_storage

[2] https://gist.github.com/mafintosh/464bb8f1451f22c9e5c5

[3] https://discuss.ipfs.io/t/ipfs-and-file-deduplication/4674




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: