> There's a reason why every major tech company has developed its own distributed filesystem
I haven't worked at FAANG, but is this a well-known fact? I've never heard of it. Unless they're referring to things like S3? Are these large corps running literal custom filesystem implementations?
Not sure about everyone, but probably yes. Imo the killer feature is durability and replication. You can use single disks if you don't care about data loss, but once you need to start replicating data you need a distributed filesystem.
Tectonic is Facebooks, Google's is Colossus. I'm not sure about the others.
It’s specialized knowledge, hard to do “correctly” (read posix here) but obtainable and implementable by a small team if you pick your battles right. Also supporting very specific use cases helps a lot.
It’s also pretty easy to justify as the hardware and software from vanguard tech companies is outrageously expensive. I used to develop software for a blue colored distributed filesystem.
I haven't worked at FAANG, but is this a well-known fact? I've never heard of it. Unless they're referring to things like S3? Are these large corps running literal custom filesystem implementations?