> More efficient that what exactly? It's still far less efficient than not having to hash and run erasure coding...

Large storage systems already do this. RAID is a type of erasure coding. Various enterprise storage systems use hashing for data integrity.

Distributing the data over a larger number of systems can actually be more efficient because the percentage of erasure blocks you need goes down as the number of independent devices increases. For example, if you only have two devices and want redundancy, you need a mirror and lose 50% of your capacity. If you spread the data across 200 devices you might achieve an even higher level of resilience by sacrificing only 25% of capacity so you can lose any 50 devices without data loss. You may get this down even further by periodically checking if a device is still available and replacing it, so the number of devices you can lose can be as low as the maximum number of devices you expect to lose simultaneously.

> That only works if the attacker cannot make big gains from a single cheat after a period of building reputation. There's a reason why this isn't being used in the wild by blockchains...

Blockchains are a different thing.

If you have a GPU and to start out with everything you produce is checked, you get 50% of what the customer pays. If you have a good reputation then only e.g. 1 in 10 of your output is checked, so you get ~90% of what the customer pays, but to do that you have to accept 50% for e.g. 100 transactions.

You can now defect, but you have a 10% chance of being detected each time, so you can expect to only get to do it 10 times. So you get a 90% payment ten times without doing work, then have to go back to getting a 50% payment 100 times. 10 times 90% is way less than 100 times 40%, so if you do this you lose money.

> Yes, it costs too much to operate, and it's too cheap as a product so operators are losing money.

Do you know what the overhead of the network actually is? Trying to put it together from multiple sources seems to imply that miners get paid ~$8/TB/year but storage costs ~$2/TB/year. Which I assume I'm doing wrong somehow, because it would imply negative overhead and therefore a huge arbitrage opportunity.

I'm guessing the real number is less than 50% overhead, because there are obvious ways to do it at least that efficiently, but even that isn't huge when you can avoid expenses for marketing and customer support. Which implies that the problem is this:

> The only reason why there's an offering at all is that some people invested lots of money on hardware in 2021 when the token price was 50 times higher (but then the storage cost was prohibitive).

Which is a self-solving problem. The unprofitable providers go out of business until the price makes it profitable. But that seems like it should happen quicker than this if the profitability isn't there, because storage is fungible. Even if you bought a bunch of drives to do this when the price was higher, you could sell them and go put the money in a traditional investment. Or if you're speculating on the value of Filecoin going up, sell your storage and use the money to buy Filecoin. So the people still doing it are presumably turning a profit even at current prices, whether through economies of scale or because they had "free" storage to use.

> Barely anyone using it despite a price so low that it doesn't even allow operators to break even, how is that supposed to be a success?

It causes very inexpensive storage to be available, which is useful.

> Large storage systems already do this. RAID is a type of erasure coding. Various enterprise storage systems use hashing for data integrity.

> Distributing the data over a larger number of systems can actually be more efficient because the percentage of erasure blocks you need goes down as the number of independent devices increases. For example, if you only have two devices and want redundancy, you need a mirror and lose 50% of your capacity. If you spread the data across 200 devices you might achieve an even higher level of resilience by sacrificing only 25% of capacity so you can lose any 50 devices without data loss. You may get this down even further by periodically checking if a device is still available and replacing it, so the number of devices you can lose can be as low as the maximum number of devices you expect to lose simultaneously.

You're mixing things up so bad it's hard to correct… For starter regarding filecoin the number of nodes you must expect to lose is “almost all of them” because they can stop operating if the economics isn't even good enough to cover their OpEx (They seem to be fine not covering the CapEx for now, but who knows for how long). It's almost like putting all your data in a datacenter owned by a nearly broke provider: if they go bankrupt you're screwed so you need a plan B.

> You can now defect, but you have a 10% chance of being detected each time, so you can expect to only get to do it 10 times. So you get a 90% payment ten times without doing work, then have to go back to getting a 50% payment 100 times. 10 times 90% is way less than 100 times 40%, so if you do this you lose money.

Again, you're mixing things up. The problem isn't that a node could defect and not do the work (this is assuming non-byzantine fault tolerance), the problem is that a node could voluntarily fuck up the calculations when/if it advantages them. And it could be far less than 10% of the time while still being a nuisance. I don't need to fuck up 10% of the back-propagation calculations in a neural-network training to make it completely unusable/to make the person training it spend way more resource than they should in the training process (which gets me more usage as a node operator).

Adversarial threat modeling is hard, I work with people who do this on a daily basis and I can clearly tell you're oversimplifying things a lot.

> Which is a self-solving problem. The unprofitable providers go out of business until the price makes it profitable. But that seems like it should happen quicker than this if the profitability isn't there, because storage is fungible. Even if you bought a bunch of drives to do this when the price was higher, you could sell them and go put the money in a traditional investment. Or if you're speculating on the value of Filecoin going up, sell your storage and use the money to buy Filecoin. So the people still doing it are presumably turning a profit even at current prices, whether through economies of scale or because they had "free" storage to use.

And yet, it hasn't self-solved itself since 2021… The problem is that node operators get paid roughly what they spend in OpEx, and their capital is essentially illiquid[1] so there's no good reason to stop operating. Of course now the only reason for them to keep invest is the hope that the token price increases again, but because this is crypto it is fueled by the “fantasy of the bull run”, not by an expected uptick in usage (which interestingly enough isn't happening even though the storage price is very cheap).

> It causes very inexpensive storage to be available, which is useful.

Yet barely anyone uses it, which empirically question its “usefulness”.

[1] and I don't get were you got the idea that storage was “fungible”, the failure rate going up exponentially over time makes storage a poor fit for the second-hand market, especially if people know that you've been running stressful proof of space-time on it, and if you're trying to fire-sale a Petabyte of storage, chances are high that people will figure that out

> For starter regarding filecoin the number of nodes you must expect to lose is “almost all of them” because they can stop operating if the economics isn't even good enough to cover their OpEx

You don't expect to lose "almost all of them" at the same time. Even if the price crashes, you would expect capacity to go down over a period of days or weeks, not minutes. And then if a fraction of the data is lost but is less than the number of erasure blocks, you promptly reconstruct it and put it on a different node.

Meanwhile you would expect an equilibrium here. The price going down forces some providers out of the market, but providers leaving the market brings the price back up. As long as the customer is offering as much as some providers need to stay in the market, somebody is hosting the data. That only stops if the customer won't bid what the providers need to get, at which point the customer transfers their data out of the system so they can stop paying more than they're willing to.

Now, you can screw this up if you make your system sufficiently convoluted so the price signal doesn't make it from the customer to the provider or vice versa, and I'm not familiar enough with the specific implementation in Filecoin to comment, but screwing that up isn't inherently necessary for this category of system.

> the problem is that a node could voluntarily fuck up the calculations when/if it advantages them.

Which is why you duplicate some of them at random, and don't tell them when you're going to do it. The calculations are deterministic. If you distribute one to two random nodes and they don't get the same result, but they've each signed their own result, now you know one of them defected and can prove which one by doing that calculation yourself or doing some other potentially expensive operation that only happens when there is an inconsistency. At which point the defector is found out, you can prove it, and their reputation is in ashes.

> Yet barely anyone uses it, which empirically question its “usefulness”.

It seems to be storing more than a exabyte of data for someone.

> their capital is essentially illiquid[1]

> [1] and I don't get were you got the idea that storage was “fungible”, the failure rate going up exponentially over time makes storage a poor fit for the second-hand market, especially if people know that you've been running stressful proof of space-time on it, and if you're trying to fire-sale a Petabyte of storage, chances are high that people will figure that out

It's fungible because a used hard drive is a commodity product with a wide customer base. That new ones cost more than the used ones doesn't make it not a commodity; you could have bought the used ones to begin if you're content to continue running them at their current age.

Let me know if you're aware of some place you can buy working >=16TB drives, used or otherwise, for less than ~$100 each in 2023.

> Meanwhile you would expect an equilibrium here. The price going down forces some providers out of the market, but providers leaving the market brings the price back up.

Bringing the XIXth century equilibrium economics here is kind of hilarious, when it has been far out of the equilibrium price for the past two years. Node runners are already losing money, and have been doing so since the end of the bull run. The value of the FIL token isn't so much derived from an equilibrium in the supply and demand for storage, it's driven by the supply and demands of the coin on the crypto market, and if nodes start to give up in any meaningful fashion over the course of a few days or weeks, the crypto market will likely negatively react, driving the price of the token even lower. In the crypto markets, Keynes' animal spirit is in charge, nobody makes rational utility calculation.

> At which point the defector is found out, you can prove it, and their reputation is in ashes.

If I can make more money out of a single adversarial attack than it costs me to build up reputation, then who cares, I'll be doing it over and over again any day.

> It seems to be storing more than a exabyte of data for someone.

It is in fact storing a exabyte of “data” for “someone”. Compare that with BitTorrent, that was used by everyone and their mom before governments started to fight it. BitTorrent was voluntary only and was a massive success. Filecoin is for profit and a failure.

> It's fungible because a used hard drive is a commodity product with a wide customer base.

Try and sell 10PB of PoST-worn-out hard drives and see how long it takes. It's far from liquid.

> The value of the FIL token isn't so much derived from an equilibrium in the supply and demand for storage, it's driven by the supply and demands of the coin on the crypto market, and if nodes start to give up in any meaningful fashion over the course of a few days or weeks, the crypto market will likely negatively react, driving the price of the token even lower.

The reason for this is that the price of FIL was initially too high for the amount of customer demand for storage it currently has, resulting in oversupply. But you only lose data as a result of sudden undersupply. If the network could lose 90% of its capacity over a month and still store all of the data it currently does, and then that happens, so what?

Whereas if it actually lost enough capacity to create scarcity given the existing demand for storage, then demand for storage would drive the price of the coin back up, right?

> If I can make more money out of a single adversarial attack than it costs me to build up reputation, then who cares, I'll be doing it over and over again any day.

How are you going to do that with AI training or something? As soon as you get caught once, people go back and retroactively verify everything you've previously done, and then you not only lose any payment received for each calculation you forged, the model you screwed up gets recomputed using the money you didn't get to keep or had to stake in order to be trusted to do computations with lower frequency verification.

> Compare that with BitTorrent, that was used by everyone and their mom before governments started to fight it. BitTorrent was voluntary only and was a massive success.

BitTorrent is a great success for large, popular data. It's pretty much useless for storing anything with a low number of downloads.

> Try and sell 10PB of PoST-worn-out hard drives and see how long it takes. It's far from liquid.

Put functional 16TB hard drives on Amazon and eBay for $99.99. See how long they last. I'd guess less than six months before you've sold 10PB worth.

> If the network could lose 90% of its capacity over a month and still store all of the data it currently does, and then that happens, so what?

If the network lose 90% of capacity over a month, you'll hear about grim the future for FIL is, on every crypto newsletter. And the price would tank even more. And if the network already lost 90% of its capacity it means that the economics is already very bad for node operators, so any worsening is likely to get even more node leaving the ship. Crypto going do dust because of crowd dynamics isn't completely unheard of…

> As soon as you get caught once, people go back and retroactively verify everything you've previously done,

How can they link me to my previous identity though… I'd just discard the previous wallet after having drained the available funds and restart from a clean state.

> the model you screwed up gets recomputed using the money you didn't get to keep or had to stake in order to be trusted to do computations with lower frequency verification.

The stacking must end at some point, and given that I can do damage with only a fraction percent adversarial computation, I can just make sure that my probability to get caught during the stacking time isn't enough for it to get a negative expected value.

Your scheme is pathologically broken, and that's no surprise, you're not going to invent a billion dollar winning multiparty computation model as an argument on HN…

> Put functional 16TB hard drives on Amazon and eBay for $99.99. See how long they last. I'd guess less than six months before you've sold 10PB worth.

You'll need to send roughly a thousand of them, without getting bad reputation from all the disks that will break soon after the buyer receives it (because on that amount, and given the state of the disks, a lots will). Also, you're not really disagreeing with my assessment, 6 month is pretty illiquid by investment standard: it's even less liquid than real estate!

> If the network lose 90% of capacity over a month, you'll hear about grim the future for FIL is, on every crypto newsletter. And the price would tank even more.

But the supply of storage goes down, which the storage buyers now need to outbid each other for, so they need to buy the coin. I'm assuming it's also possible for the price of storage in FIL to go down as the price moves. If $1 US is 100 FIL but now providing 1TB/year of storage yields hundreds of FIL, you still earn several dollars US per TB stored.

> How can they link me to my previous identity though… I'd just discard the previous wallet after having drained the available funds and restart from a clean state.

No reputation is the same as bad reputation. To have a good reputation you have to engage in a large number of transactions which are less profitable to you because they're undergoing 100% verification. Building a good reputation allows you to make higher margins, which is valuable and therefore costly to sacrifice.

You can't transfer funds you've staked against your reputation until the buyer has had a reasonable amount of time to try to prove you defected.

> The stacking must end at some point, and given that I can do damage with only a fraction percent adversarial computation, I can just make sure that my probability to get caught during the stacking time isn't enough for it to get a negative expected value.

Suppose you have a good reputation so you only undergo verification 10% of the time at random instead of 100% of the time. You also have to hold 20 times your revenue from this transaction as collateral during the verification window, or however much is necessary to more than compensate the buyer and punish you in the event that you defected.

Now if you defect you have a 10% chance of losing 2000% of your payment. This has a negative expected value. Meanwhile it's now public that you defected and every other buyer still in the verification window is going to go back and verify 100% of their transactions with you, causing you to have a 100% chance of losing 2000% of your payment for those transactions if you defected.

> You'll need to send roughly a thousand of them, without getting bad reputation from all the disks that will break soon after the buyer receives it (because on that amount, and given the state of the disks, a lots will).

The annual failure rate for ~6 year old hard drives is ~2%. Presumably the failure rate over six months is about half that, and you have plenty of other functional drives to send replacements to satisfy the ~1% of customers who got unlucky.

> Also, you're not really disagreeing with my assessment, 6 month is pretty illiquid by investment standard: it's even less liquid than real estate!

That's only because you're trying to sell 10PB of hard drives. It's like saying shares of stock are illiquid because if you want to sell ten billion dollars of shares in the same company it might not be advisable to do it all on the same day.

And even that you could still do, if you want to solicit a large buyer, which in this context would presumably be some kind of data center.

But even supposing that it would take six months, what's your reasoning for why it has already persisted for longer than that period of time then?

> No reputation is the same as bad reputation. To have a good reputation you have to engage in a large number of transactions which are less profitable to you because they're undergoing 100% verification. Building a good reputation allows you to make higher margins, which is valuable and therefore costly to sacrifice.

This is just a balance between how much you win, and how much it costs you. If I double my earnings for sub 1% chance of being caught, then you need to have a very expensive reputation build-up to compensate for that, and this is going to put a big burden on legit providers who want to enter the system, making it even easier to cheat.

> You also have to hold 20 times your revenue from this transaction as collateral during the verification window, or however much is necessary to more than compensate the buyer and punish you in the event that you defected.

Same as above: the higher the stacking is to fend of cheaters, the less attractive it is to legit players. Also, with your scheme the “verification window” doesn't matter, since you're not going to catch me after the fact: you're going to catch me iif my adversarial transaction is being checked.

> Now if you defect you have a 10% chance of losing 2000% of your payment.

Not if me cheating allows me to make even just 223% of the legit payment.

As I said before, you're not going to design a billion-dollar scheme in this HN discussion…

Edit: I just realized that your scheme would be even more penalizing to legit node than I thought: with one random bit flip, the node would lose all their stacking and all their reputation. Talk about an expensive cosmic ray! (Or an attacker could even voluntarily send rowhammer workload to legit node in order to destroy their reputation and stacking, reducing supply and hence increasing their own margin). And I'll say it again: this stuff is HARD and you're very unlikely to find a working solution on your own in this discussion!

> The annual failure rate for ~6 year old hard drives is ~2%.

Not if you've spend the said 2 years stressing the drive in a PoST scheme. There's a reason why these schemes break the manufacturer's warranty …

> That's only because you're trying to sell 10PB of hard drives. It's like saying shares of stock are illiquid because if you want to sell ten billion dollars of shares in the same company it might not be advisable to do it all on the same day.

You'll be able to sell them. You'll take a haircut (likely less than the 70% you're talking about when reselling old hard drives), but you'll sell them in the same day anyway.

> And even that you could still do, if you want to solicit a large buyer, which in this context would presumably be some kind of data center.

Good luck selling worn-out hardware to a data center!

> But even supposing that it would take six months, what's your reasoning for why it has already persisted for longer than that period of time then?

Hodl to the moon (AKA sunk cost fallacy)… They have an illiquid asset, have swallowed the cost of the capital investment (i.e. they had little to no leverage on it) and no need to fire sell it. Also the 6 months is just your optimistic hypothesis…

> This is just a balance between how much you win, and how much it costs you. If I double my earnings for sub 1% chance of being caught, then you need to have a very expensive reputation build-up to compensate for that, and this is going to put a big burden on legit providers who want to enter the system, making it even easier to cheat.

But there is also no need to make the chance of being caught so low, because a single digit percentage of overhead is completely reasonable while still providing a significant chance of being caught.

And you could scale the verification rate with reputation, so a 1% verification rate is possible but takes a very long time, whereas a 10% verification rate is more than ten times easier to get despite still not being a prohibitively high amount of overhead.

> Same as above: the higher the stacking is to fend of cheaters, the less attractive it is to legit players.

For things like GPU computation, you're going to do a unit of computation over a matter of minutes with verification taking the same amount of time or being done in parallel, and so you do many units of computation a day.

It's not that unreasonable to ask someone to put a month's earnings at stake at any given time, which is about what you get with a 24 hour verification window and a 7% verification rate.

> Also, with your scheme the “verification window” doesn't matter, since you're not going to catch me after the fact: you're going to catch me iif my adversarial transaction is being checked.

You're not the one who chooses whether to verify it. If other people are verifying 10% of your work but then someone catches you cheating, they can prove to the others that you cheated and then everyone goes back and verifies 100% of your work which is still in the verification window -- at your expense -- and you lose even more if you cheated more than once.

> Not if me cheating allows me to make even just 223% of the legit payment.

The thing this is preventing is you claiming to do some work but actually not, e.g. someone wants to use your GPU for AI but you don't even have a GPU and just return random numbers. To know if your result is right they would have to do the same computation again so they can compare them, which doubles the cost, or more if they want higher assurances against collusion. And then they're not willing to pay you as much because some of their money has to go to that.

Also, it's 223% plus the cost of provably damaging your reputation.

> I just realized that your scheme would be even more penalizing to legit node than I thought: with one random bit flip, the node would lose all their stacking and all their reputation. Talk about an expensive cosmic ray!

You can set the penalty to whatever is necessary to deter cheating at that level of verification. It doesn't have to be that high, but it can be that high if you need it to be without imposing an unrealistic amount of overhead.

And providers who don't want to be penalized for doing the calculation wrong should operate reliable hardware with functioning error correction. This is not a bad thing to incentivize.

You might also weight the reputational harm. If you get caught once, your reputation will be harmed and lots of people will be rechecking your recent results to see if you tried to screw them too, but if it's an isolated incident you only take a small hit. Whereas if you get caught repeatedly, well, you might as well just start over.

> Or an attacker could even voluntarily send rowhammer workload to legit node in order to destroy their reputation and stacking, reducing supply and hence increasing their own margin

At which point the node is at least as likely to crash as sign an invalid result, which is already a denial of service attack you have to mitigate. For example by using ECC memory and terminating workloads that induce detectable ECC errors instead of continuing them until they induce an undetectable one and crash the machine or cause it to sign a corrupted result.

(Also, rowhammer is a huge problem and almost nobody is actually mitigating it effectively for anything. Someone needs to come up with a generic solution for it before someone else starts using it for widespread exploitation or we're going to have a bad time regardless of what kind of reputation systems are in use.)

> Not if you've spend the said 2 years stressing the drive in a PoST scheme. There's a reason why these schemes break the manufacturer's warranty ...

Do you have some data to back up this claim? Drives are routinely used for heavy database workloads and reliable drive models still last for multiple years.

It seems evident that they're at least reliable enough to continue operating under that workload since that's what they have done instead of failing en masse and causing the storage capacity of the network to decline, given the assumption that the price is too low to justify anyone replacing them.

> You'll be able to sell them. You'll take a haircut (likely less than the 70% you're talking about when reselling old hard drives), but you'll sell them in the same day anyway.

You can sell the hard drives the same day too if you're willing to provide a sufficient discount from the market price. But there is rarely a good reason to do this, because the discount you'd have to provide is more than the time value of money in spreading the sales over somewhat more time.

> Good luck selling worn-out hardware to a data center!

Data centers run hardware until the resource consumption in power and space exceeds the cost of newer hardware, or until it dies. Reliability is just a number in an equation that tells you how much redundancy you have to operate with.

> Hodl to the moon (AKA sunk cost fallacy)

Those are two different things. If they want to hold the coin they'd have more of it to hold if they sell their hardware and use the money to buy the coin.

More likely they're expecting that others will quit and lower the supply to fall in line with demand (or more optimistically that demand will increase) so they can go back to making a profit, but since they all have that incentive they hodl until somebody blinks first, and lose if nobody does.

In the meantime their possibly irrational optimism provides for cheap storage.

Cranks are a scourge…