Async replication implemented with 1 second of data loss expected when the primary goes down.
> However, we don't write every individual LTX file to object storage immediately. The latency is too high and it's not cost effective when you write a lot of transactions. Instead, the LiteFS primary node will batch up its changes every second and send a single, compacted LTX file to LiteFS Cloud. Once there, LiteFS Cloud will batch these 1-second files together and flush them to storage periodically.
That's not true. LiteFS itself is a distributed database so you have redundancy within your cluster outside of LiteFS Cloud. A typical setup is to run two candidate nodes within a single region so they have low replication latency and then adding read-only nodes in other regions. Transactions are replicated immediately after commit so a write on the primary will be sent to the other candidate node within a millisecond or two. It is async replication but the data loss window when you're running a setup like this is quite small.
I see. So it’s 1 second plus time to write to S3 as the max data loss window.
Which you can experience if your network connection, etc to the secondaries go bad, particularly if you aren’t alerting about it and fixing the situation.
To try to avoid any data loss one would need synchronous replication. There is a SQLite fork that uses RAFT.
> However, we don't write every individual LTX file to object storage immediately. The latency is too high and it's not cost effective when you write a lot of transactions. Instead, the LiteFS primary node will batch up its changes every second and send a single, compacted LTX file to LiteFS Cloud. Once there, LiteFS Cloud will batch these 1-second files together and flush them to storage periodically.