tl;dr2 when you read "Paxos" or "distributed"-anything just assume it's a bunch of nerds arguing about theoretical problems inside a box
Recap of those wiki pages:
- Consensus is many different ways to solve similar problems, all of which involve agreeing on a solution, to different degrees.
- The two-generals' problem is a proved unsolvable problem that says you can't know what two generals are going to do if you have no way to prove communication between them is valid (or happens at all).
- Byzantine faults are basically the same thing, but 10 people instead of 2.
- Paxos is a bunch of ways of proving various degrees of different kinds of consensus with different uses that works most of the time. (also known as "the algorithm family that comp sci majors keep trying to improve on, but then a genius points out that any changes make it work differently, and therefore must suck")
Why do we care? So you can use 5 servers all around the world, store random messages on them, and be sure that a bunch of them have the information you want, that it is correct, that it will still be available somewhere if one of them goes down, and that if one comes back up with shitty data, it will get corrected and become good data again.
The reason why this is hard is we assume that the messages:
1. have no CRC (integrity, "this data is not corrupt")
2. or digital signature (integrity+authenticity, "this message is not corrupt and definitely came from Bob")
3. and that even if they did, it could have been 3a. "wrong" before it was signed, or 3b. "faked" so that we can't tell when a CRC or sign is correct,
4. or the message never arrived,
5. or that random valid messages have been saved on one server and that when it rejoins the group it now has data that, regardless of it being good or bad, we don't want in the rest of the pool of servers because now the rest have to incorporate it, and what if there are conflicts
The Paxos algorithm family seeks to solve most of this in one go, with exceptions.
Do you need Paxos to solve all those problems? No. Do you need it to solve those problems in the theoretical boundaries of comp sci nerds? Yes. Do you need to know how it works? No. Are there situations where a Paxos network can simply stop working? Yes. Do you need to use Paxos for something you're working on? Probably not. Is it possible Paxos will not solve your problem? Yes. And do you still need plans to back up and restore your data in the event of catastrophic failure? Definitely.
tl;dr2 when you read "Paxos" or "distributed"-anything just assume it's a bunch of nerds arguing about theoretical problems inside a box
Recap of those wiki pages:
- Consensus is many different ways to solve similar problems, all of which involve agreeing on a solution, to different degrees.
- The two-generals' problem is a proved unsolvable problem that says you can't know what two generals are going to do if you have no way to prove communication between them is valid (or happens at all).
- Byzantine faults are basically the same thing, but 10 people instead of 2.
- Paxos is a bunch of ways of proving various degrees of different kinds of consensus with different uses that works most of the time. (also known as "the algorithm family that comp sci majors keep trying to improve on, but then a genius points out that any changes make it work differently, and therefore must suck")
Why do we care? So you can use 5 servers all around the world, store random messages on them, and be sure that a bunch of them have the information you want, that it is correct, that it will still be available somewhere if one of them goes down, and that if one comes back up with shitty data, it will get corrected and become good data again.
The reason why this is hard is we assume that the messages:
1. have no CRC (integrity, "this data is not corrupt")
2. or digital signature (integrity+authenticity, "this message is not corrupt and definitely came from Bob")
3. and that even if they did, it could have been 3a. "wrong" before it was signed, or 3b. "faked" so that we can't tell when a CRC or sign is correct,
4. or the message never arrived,
5. or that random valid messages have been saved on one server and that when it rejoins the group it now has data that, regardless of it being good or bad, we don't want in the rest of the pool of servers because now the rest have to incorporate it, and what if there are conflicts
The Paxos algorithm family seeks to solve most of this in one go, with exceptions.
Do you need Paxos to solve all those problems? No. Do you need it to solve those problems in the theoretical boundaries of comp sci nerds? Yes. Do you need to know how it works? No. Are there situations where a Paxos network can simply stop working? Yes. Do you need to use Paxos for something you're working on? Probably not. Is it possible Paxos will not solve your problem? Yes. And do you still need plans to back up and restore your data in the event of catastrophic failure? Definitely.
The only paper that I know of currently that I would recommend anyone to read on byzantine failures and their solutions is this one: https://www.cs.indiana.edu/classes/p545/post/lec/fault-toler...
(Granted, I don't know what I am talking about, so take all that with a grain of salt)