Hacker Newsnew | past | comments | ask | show | jobs | submit | more PhantomHour's commentslogin

> Does allow bots to access my information prevent other people from accessing my information? No.

Yes it does, that's the entire point.

The flood of AI bots is so bad that (mainly older) servers are literally being overloaded and (newer servers) have their hosting costs spike so high that it's unaffordable to keep the website alive.

I've had to pull websites offline because badly designed & ban-evading AI scraper bots would run up the bandwidth into the TENS OF TERABYTES, EACH. Downloading the same jpegs every 2-3 minutes into perpetuity. Evidently all that vibe coding isn't doing much good at Anthropic and Perplexity.

Even with my very cheap transfer racks up $50-$100/mo in additional costs. If I wanted to use any kind of fanciful "app" hosting it'd be thousands.


I'm still very confused by who is actually benefitting from the bots; from the way they behave it seems like they're wasting enormous amounts of resources on both ends for something that could have been done massively more efficiently.


That's a problem with scrapers, not with AI. I'm not sure why there are way more AI scraper bots now than there were search scraper bots back when that was the new thing. However that's still an issue of scapers and rate limiting and nothing to do with wanting or not wanting AI to read your free and open content.


This whole discussion is about limiting bots and other unwanted agents, not about AI specifically (AI was just an obvious example)


"Before they were like this"

I would like to remind you that Facebook got it's start as a sex pest website.


> You really do have to account for why this is mainly happening in industries that are adopting AI

Correlation is not causation. The original research paper does not prove a connection.

> I doubt very much that these three Stanford professors would be blindsided by the concept of rates and tarriffs.

They are nonetheless subject to publish or perish pressure and have strong incentives to draw publishable attention-grabbing results even where the data is inconclusive.


> I doubt very much that these three Stanford professors would be blindsided by the concept of rates and tarriffs.

Digital Economy Lab and the Stanford Institute for Human-Centered Artificial Intelligence

I fully expect that these professors would be blindsided by even the most rudimentary real world economics.


Tariffs are just a massive government revenue generating consumption tax on particular industries. We would expect unemployment among the young trying to enter those industries to be hit hardest.


[flagged]


i'm curious who you think pays american tarrifs


You first


Everyone pays mate


> Do you understand that American employers don't have to pay American tariffs?

Except they do, if their raw materials, tools, etc., are imported.


It's a business model problem; The "Uber" business model relies on a monopoly.

The business model is 1) "Have artificially low prices to push all competing business into bankrupty", 2) "Now that we're a monopoly, raise prices massively", 3) Massive profit, so long as no government starts doing anything about the fact that both steps #1 and #2 are illegal.

That business model fails the moment you have multiple startups dumping the market, none can move to step #2 because they'd bleed all their users to whichever competitor is still in step #1.


In part it'll be Europe, though VC in the "throw money into a fire" style does/did exist.

But VCs, especially in those days, bordered on antipathy for sensible business plans. They didn't want small businesses that would turn profitable quickly and grow sustainably. They wanted something with infinite growth ASAP that they could pump-and-dump on Big Tech or IPO suckers.


> These critiques of zirp never explain what should have been done differently.

Controversially: They don't have to. The point of this piece (and similar ones) isn't "how we should treat the next similar-to-2008 financial crisis", it's that the current desire to "RETRVN TO ZIRP" is dangerous.

Perhaps there truly was no better way to respond to the Great Recession. There's a pretty good case to be made that the recovery has been a wonderous success of modern central banking. That doesn't change the fact that ZIRP had downsides, big ones.

> If the Fed had actually made money "too cheap" inflation would have kicked in much earlier.

I'm not terribly sold on this economic theory, so do take that in mind for the rest of this comment.

One thing to consider: While there was shockingly little inflation in general goods and services, there has been pretty notable asset price inflation. P/E ratios have been steadily creeping up since 2010. The real estate market's supply-shortage has turbocharged it's inflation.

> Ultimately money is neutral in the long run.

This is true, but not particularly useful; It's a very "long" "long run". The conceit of modern central banking is to "smooth out" that long term trend, and we have many examples where (even non-modern) monetary policy can screw up a country.

The big problem with Trump's desire to hit the gas on the economy and slam interest rates into the floor is that this just very clearly does not work. At worst he'll quickly find himself in the situation Erdogan got himself, at best (that is, "best Trump seizes the fed" scenario) the US will find itself in an asset bubble economy like Japan.

And unlike Japan and Turkey, the US has a lot more to lose. Pension funds make up a sizable portion of the wealth and spending in the US. If the stock market were to take a hit similar to the Dotcom bubble crash or "Lost Decade(s)", pensions will need to be adjusted downwards, to disastrous and self-reinforcing economic consequences. (Another fun layer to this is that those pensions tend to be supplemented by real estate assets, which are not in a "true" bubble but will most certainly collapse in a steep recession.)


Then you add in the fact pensions are invested into Private Equity, and it’s glaringly obvious how fragile (and already broken) the US economy is.

Returning to ZIRP is bad, as is removing the Fed’s independence, as is allowing PE to continue operating unchecked, as is a whole bunch of other stuff (over regulation of small businesses, under regulation of corporate behemoths, over reliance on government assistance programs by workers of for-profit companies due to low wages, the precarity of gig work, the displacement of educated workers by automation while simultaneously dismantling social safety nets, the student debt crisis, the housing crisis, the auto crisis, infrastructure crisis, etc, etc).

As you pointed out (and the initial detractor ignores), it’s not that ZIRP itself was bad in theory, but rather that a return to it - knowing the harms it caused - is bad, and that there is no outcome of the Fed losing independence that doesn’t end with the wholesale demolition of the foundation of the global economy, that being the US Dollar and Economic engine lifting all boats through political independence.


> Like it or not, employers will be forced to cooperate with you to get access to the talent pool you attract.

Except for the problem of "talent will be forced to seek out employers, no matter how shitty or stupid the latter behaves, because they'll starve and die within a matter of weeks or months while understaffed companies can survive for years."

Doubly so in tech where the combination of A) A huge hiring spree during covid & following layoffs has created a glut in applicants, B) Economic malaise is slowing the economy, and C) Companies are being irrationally hestitant to hire because of AI.

Ghost Jobs are fraudulent on several levels, they should be legislated out of existance. (The public company favourite of "pretending we're still growing when we're not" is very clear securities fraud.)


The entire point of the funding is the strategic benefit that Intel's fabs in the US provide. That's the thing the US would get out of it.

Also consider: Who is the $10 billion coming from? Because it's not Intel. Intel just prints the shares out of thin air. This is Trump stealing $10 billion directly from Intel's shareholders, who get nothing in return because Trump is legally required to disburse the CHIPS Act money.


It's also just a complete misunderstanding in how a governments vehicle for investment works compared to a person's.

A person invests in a stock, hopes it goes up, and makes a profit (or loss) when they sell. That money isn't real until a sale occurs.

A government gives a grant and gets a return tomorrow and every single day of the company's existence as well as every single day each employee exists (even if the company doesn't). If a company exists, it pays corporate taxes. If that company has employees (as every company does), it pays payroll taxes. If that company has employees, they pay income taxes and various forms of consumption taxes.

IDK why we act like a grant is akin to lighting money on fire. Governments don't give out grants for nothing. They're still benefiting from it. Sure, the vehicles for return are different from a standard investment but also a government isn't a person. Well I should take that last part back. A government is a person in an autocracy (monarchy/dictatorship/theocracy/etc), but that shouldn't even be on the table.


> that shouldn't even be on the table

I wish that were so.


Ditto.

It's also just crazy to see nationalization being packaged as capitalism. Those screaming for privatization while nationalizing. It's not even being done covertly either


The way around that is that is for LLM-based tools to run a regular search engine query in the background and feed the results of that in alongside the prompt. (Usually a two-step process of the LLM formulating the query, then another pass on the results)

The used results can then have their link either added to the end result separately, guaranteeing it is correct, or added to the prompt and "telling the LLM to include it", which retains a risk of hallucination, yes.

Common to both of these is the failure mode that the LLM can still hallucinate whilst "summarizing" the results, meaning you still have no guarantee that the claims made actually show up in the results.


> The way around that is that is for LLM-based tools to run a regular search engine query in the background and feed the results of that in alongside the prompt. (Usually a two-step process of the LLM formulating the query, then another pass on the results)

Would the LLM-based tool be able to determine that the top results are just SEO-spam sites and move lower in the list, or just accept the spam results as gospel?


This is an extremely tricky question.

The practical and readily-observable-from-output answer is "No, they cannot meaningfully identify spam or misinformation, and do indeed just accept the results as gospel"; Google's AI summary works this way and is repeatedly wrong in exactly this way. Google's repeatedly had it be wrong even in the adcopy.

The theoretical mechanism is that the attention mechanism with LLMs would be able to select which parts of the results are fed further into the results. This is how the model is capable of finding parts of the text that are "relevant". The problem is that this just isn't enough to robustly identify spam or incorrect information.

However, we can isolate this "find the relevant bit" functionality away from the rest of the LLM to enhance regular search engines. It's hard to say how useful this is; Google has intentionally damaged their search engine and it may simply not be worth the GPU cycles compared to traditional approaches, but it's an idea being widely explored right now.


The only thing that can solve the misinformation from a bad LLM is the misinformation from a good LLM... with a gun.


>The way around that is that is for LLM-based tools to run a regular search engine query in the background and feed the results of that in alongside the prompt.

Hardly better, as soon those "search engine results" would be AI slop themselves, including actual published papers (phoned-in by using AI, and "peer reviewed" by using AI from indifferent reviewers)


One thing I'm missing in the full report is what a 'median prompt' actually looks like. How many tokens? What's the distribution of prompt sizes like? Is it even the same 'median prompt' between 2024 and 2025?

The numbers are cute but we can't actually do anything with them without those details. At least an average could be multiplied by the # of queries to get the total usage.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: