> [Ng] said that the “bad idea that AI could make us go extinct” was merging with the “bad idea that a good way to make AI safer is to impose burdensome licensing requirements” on the AI industry.

> “There’s a standard regulatory capture playbook that has played out in other industries, and I would hate to see that executed successfully in AI.”

> “Just to be clear, AI has caused harm. Self-driving cars have killed people. In 2010, an automated trading algorithm crashed the stock market. Regulation has a role. But just because regulation could be helpful doesn’t mean we want bad regulation.”

[1]: https://www.afr.com/technology/google-brain-founder-says-big...

[2]: https://web.archive.org/web/20231030062420/https://www.afr.c...

But imagine all the money bigco can make by crippling small startups from innovating and competing with them! It's for your own safety. Move along citizen.

The trouble is sometimes they don't. Or they do exist for a reason but the rules are still absurd and net harmful because they're incompetently drafted. Or the real reason is bad and the rules are doing what they were intended to do but they were intended to do something bad.

> If China builds amazing AI tech (and they will) then the rest of the world will just use it.

Not if it's banned elsewhere, or they allow people to use it without publishing it, e.g. by offering it as a service.

And it matters a lot who controls something. "AI" potentially has a lot of power, even non-AGI AI -- it can create economic efficiency, or it can manipulate people. If an adversarial entity has greater economic efficiency, they can outcompete you -- the way the US won the Cold War was essentially by having a stronger economy. If an adversarial entity has a greater ability to manipulate people, that could be even worse.

> If you want to live in China, go live in China. I assure you you will not find it to be the law less free hold of “anything goes” that you somehow imagine.

But that's precisely the issue -- it's not an anarchy, it's an authoritarian competing nation state. We have to be better than them so the country that has an elected government and constitutional protections for human rights is the one with an economic advantage, because it isn't a law of nature that those things always go together, but it's a world-eating disaster if they don't.

Ok.

…but if you have a law and you’re opposed to it on the basis that “China will do it anyway”, you admit that’s stupid?

Shouldn’t you be asking: does the law do a useful thing? Does it make the world better? Is it compatible with our moral values?

Organ harvesting.

Stem cell research.

Human cloning.

AI.

Slavery.

How can anyone stand there and go “well China will do it so we may as well?”

In an abstract sense this is a fundamentally invalid logical argument.

Truth on the basis of arbitrary assertion.

It. Is. False.

Now, certainly there is a degree of naunce with regard to AI specifically; but the assertion that we will be “left behind” and “out competed by China” are not relevant to the discussion on laws regarding AI and AI development.

What we do is not governed by what China may or may not do.

If you want to win the “AI race” to AGI, then investment and effort is required, not allowing an arbitrary “anything goes” policy.

China as a nation is sponsoring the development of its technology and supporting its industry.

If you want want to beat that, opposing responsible AI won’t do it.

There are plenty of technologies that absolutely do not "make the world better" but unfortunately must get built because humans are shitty to each other. Weapons are the obvious one, but not the only one. Often countries pass laws to encourage certain technologies or productions so as not to get outcompeted or outproduced by other countries.

The argument here about AI is exactly this sort of argument. If other countries build vastly superior AI by have fewer developmental restrictions, then your country maybe both at a military disadvantage but also at an economic disadvantage because you can be easily outproduced by countries using vastly more efficient technology.

You must balance all the harms and benefits when making laws, including external to the country issues.

That depends on what "it" is. If it's slavery and the US but not China banning slavery causes there to be half as much slavery in the world as there would be otherwise, it would be stupid.

But if it's research and the same worldwide demand for the research results are there so you're only limiting where it can be done, which only causes twice as much to be done in China if it isn't being done in the US, you're not significantly reducing the scope of the problem. You're just making sure that any benefits of the research are in control of the country that can still do it.

> Now, certainly there is a degree of naunce with regard to AI specifically; but the assertion that we will be “left behind” and “out competed by China” are not relevant to the discussion on laws regarding AI and AI development.

Of course it is. You could very easily pass laws that de facto prohibit AI research in the US, or limit it to large bureaucracies that in turn become stagnant for lack of domestic competitive pressure.

This doesn't even have anything to do with the stated purpose of the law. You could pass a law requiring government code audits which cost a million dollars, and justify them based on any stated rationale -- you're auditing to prevent X bad thing, for any value of X. Meanwhile the major effect of the law is to exclude anybody who can't absorb a million dollar expense. Which is a bad thing even if X is a real problem, because that is not the only possible solution, and even if it was, it could still be that the cure is worse than the disease.

Regulators are easily and commonly captured, so regulations tend to be drafted in that way and to have that effect, regardless of their purported rationale. Some issues are so serious that you have no choice but to eat the inefficiency and try to minimize it -- you can't have companies dumping industrial waste in the river.

But when even the problem itself is a poorly defined matter of debatable severity and the proposed solutions are convoluted malarkey of indiscernible effectiveness, this is a sure sign that something shady is being evaluated.

A strong heuristic here is that if you're proposing a regulation that would restrict what kind of code an individual could publish under a free software license, you're the baddies.

…

> A strong heuristic here is that if you're proposing a regulation that would restrict what kind of code an individual could publish under a free software license, you're the baddies.

Sure.

…but those things will change the way development / progress happens regardless of what China does.

“We have to do this because China will do it!” is a harmful trope.

You don’t have to do anything.

If you want to do something, then do it, if it makes sense.

…but I flat out reject the original contention that China is a blanket excuse for any fucking thing.

Take some darn responsibility for your own actions.

Yes it is... Where the hell would you get the impression we don't change how we govern and invest based on what China does, is doing, or might be doing? Do you really think nations don't adjust their behavior and laws based on other counties real or perceived? I can't imagine you're that ignorant.

> If you want want to beat that, opposing responsible AI won’t do it.

Not opposing it guarantees you lose though.

Maybe the laws around AI will drastically impact the ability of startups to compete with foreign competitors.

…but I can’t see that being likely.

It seems to me that restricting chip technology has a much much more significant impact, along with a raft of other measures which are already in place.

All I can see when I look closely at arguments from people saying this kind of stuff is people who want to make deep fakes, steal art and generate porn bots crying about it, and saying it not fair other people (eg. Japan, where this has been ruled legal, China for who knows what reason, mostly ignorance) are allowed to do it.

I’m not sympathetic.

I don’t believe that makes any difference to the progress on AGI.

I don’t care if China out competes other countries on porn bots (I don’t think they will; they have a very strict set of rules around this stuff… but I’ll be generous and include Japan which probably will).

You want the US to get AGI first?

Well, explain specifically how you imagine open source (shared with the world) models, and open code sharing vs. everything being locked away in a Google/Meta sandbox helps?

Are you sure you’re arguing for the right side here? Shouldn’t you be arguing that the models should be secret so China can’t get them?

Or are you just randomly waving your arms in the air about China without having read the original article?

What are you even arguing for? Laws are bad… but sharing with China is also bad… but having rules about what you do is bad… but China will do it anyway… but fear mongering and locking models away in big corporations behind apis is bad… but China… or something…

????

It’s really not a compelling argument.

Why not? It's what happens with other things.

> It seems to me that restricting chip technology has a much much more significant impact, along with a raft of other measures which are already in place.

Restricting chip technology is useless and the people proposing it are foolish. Computer chips are generic technology and AI things benefit from parallelism. The only difference between faster chips and more slower chips is how much power they use, so the only thing you get from restricting access to chips is more climate change.

> All I can see when I look closely at arguments from people saying this kind of stuff is people who want to make deep fakes, steal art and generate porn bots crying about it, and saying it not fair other people (eg. Japan, where this has been ruled legal, China for who knows what reason, mostly ignorance) are allowed to do it.

The problem is not that people won't be able to make porn bots. They will make porn bots regardless, I assure you. The problem is that the people who want to control everything want to control everything.

You can't have a model with boobs in it because that's naughty, so we need a censorship apparatus to prevent that. And it should also prevent racism, somehow, even though nobody actually agrees how to accomplish that. And it can't emit foreign propaganda, defined as whatever politicians don't like. And now that it has been centralized into a handful of megacorps, they can influence how it operates to their own ends and no one else can make one that works against them.

Now that you've nerfed the thing, it's worse at honest work. It designs uncomfortable apparel because it doesn't understand what boobs are. You ask it how something would be perceived by someone in a particular culture and it refuses to answer, or lies to you because of what the answer would be. You try to get it to build a competing technology to the company that operates the thing and all it will do is tell you to use theirs. You ask it a question about the implications of some policy and its answer is required to comply with specific politics.

> Well, explain specifically how you imagine open source (shared with the world) models, and open code sharing vs. everything being locked away in a Google/Meta sandbox helps?

To improve it you can be anyone anywhere vs. to improve it you have to work for a specific company that only employs <1% of the people who might have something to contribute. To improve it you don't need the permission of someone with a conflict of interest.

> Are you sure you’re arguing for the right side here? Shouldn’t you be arguing that the models should be secret so China can’t get them?

China is a major country. It will get them. The only question is if you will get them, in addition to China and Microsoft. And to realize the importance of this, all you have to ask is if all of your interests are perfectly aligned with those of China and Microsoft.

When we ask it questions, an AI can tailor its answers to change peoples opinions and how people think. They would have the power to influence elections, our values, our sense of right and wrong.

That's before we start allowing AI to just start making purchasing decisions for us with little or no oversight.

The only answer I see is for us all to have our own AI's that we have trained, understand, and trust. For me this means it runs on my hardware and answers only to me. (And not locked behind regulation)

"Don't worry if our adversary develops nuclear weapons and we won't - it's OK we'll just use theirs"

Beneath this comment is hidden a truth that there is AI which can be used beneficially, AI which can be used detrimentally, AI which can be weaponized in warfare, and AI which can be used defensively in warfare. Discussions about policy and regulation should differentiate these, but also consider implications of how this technology is developed and for what purpose it could be employed.

We should definitely be developing AI to combat AI as it will most certainly be weaponized against us with greater frequency in the near future.

So it's not just pure warfare systems that are risky for us but everything.

The problem is what the Powers-That-Be say and what they do are not in alignment.

We are now, after much long-time pressure from everyone not in power saying that being friendly with China doesn't work, waging a cold war against China and presumably we want to win that cold war. On the other hand, we just keep giving silver platter after silver platter to China.

So do we want the coming of Pax Sino or do we still want Pax Americana?

If we defer to history, we are about due for another changing of the guard as empires generally do not last more than a few hundred years if that, and the west seems poised to make that prophecy self-fulfilling.

Here's the thing: the US didn't win the OG Cold War by being, as 'AnthonyMouse puts it upthread, "the country that has an elected government and constitutional protections for human rights" and "having a stronger economy". It won it by having a stronger economy, which it used to fuck half of the world up, in a low-touch dance with the Soviets that had both sides toppling democratic governments, funding warlords and dictatorships, and generally doing the opposite of protecting human rights. And at least through a part of that period, if an American citizen disagreed, or urged restraint and civility and democracy, they were branded a commie mutant spy traitor.

My point here isn't to pass judgement on the USA (and to be clear, I doubt things would've been better if the US let Soviets take the lead). Rather, it's that when we're painting the current situation as the next Cold War, then I think people have a kind of cognitive dissonance here. The US won the OG Cold War by becoming a monster, and not pulling any punches. It didn't have long discussions about how to safely develop new technologies - it just went full steam ahead, showered R&D groups with money, while sending more specialists to fuck up another country to keep the enemy distracted. This wasn't an era known for reasoned approach to progress - this was the era known for designing nuclear ramjets with zero shielding, meant to zip around the enemy land, irradiating villages and rivers and cities as they fly by, because fuck the enemy that's why.

I mean, if it is to happen, it'll happen. But let's not pretend you can keep Pax Americana by keeping your hands clean and being a nice democratic state. Or that whether being more or less serious about AI safety is relevant here. If it becomes a Cold War, both sides will just pull all the stops and rush full-steam to develop and weaponize AGI.

--

EDIT - an aside:

If the history of both sides' space programs is any indication, I wouldn't be surprised to see the US building a world-threatening AGI out of GPT-4 and some duct tape.

Take for example US spy satellites - say, the 1960s CORONA program. Less than a decade after Sputnik, no computers, with engineering fields like control theory being still under development - but they successfully pulled off a program that involved putting analog cameras in space on weird orbits, which would make ridiculously high-detail photos of enemy land, and then deorbit the film canisters, so they could be captured mid-air by a jet plane carrying a long stick. If I didn't know better, I'd say we don't have the technology today to make this work. The US did it in the 1960s, because it turns out you can do surprisingly much with surprisingly little, if you give creative people infinite budget, motivate them with basic "it's us vs. them" story, and order them to win you the war.

As impressive as such feats were (and there were plenty more), I don't think we want to have the same level of focus and dedication applied to AI - if that's a possibility, then I fear we've crossed the X-risk threshold already with the "safe" models we have now.

These terms, the Western companies agreed to to gain a piece of the juicy Chinese market. And the Chinese did it because they had the rare power to stop Western companies from just coming and draining resources, in the colonial manner the West usually operates.

Building on this, China has now surpassed the West on much development. Electric cars, solar technology, cell phone towers are now much more advanced in China.

The West started shifting production to China for immense cost savings, over 40 years ago. At the time, China had almost NO market, and no (what the West called, at the time) "middle class". China was mostly agrarian, and had very little manufacturing base.

There was nothing "juicy" for the West, market wise. At all.

Over the last 40 years, China's economy has prospered, grown, again mostly due to the West's user of Chinese labour. Virtually the entire manufacturing base that China has right now, exists because Western expertise, skill, and capabilities helped Chinese factories, and workers, come online and train in Western production methods.

Prior to 40 years ago, everyone except the British couldn't have cared less for China, and the British indeed had Hong Kong.. something pre-existent from THEIR colonial days. The British could have retained Hong Kong, but as agreed did turn it over to China at the turn of the century. No, China had no capability to enforce that, not back around the year 2000.

Note that the colonial days of "the West" makes little sense. Many Western nations were not colonialists, and the US is actually a breakaway colony, and has worked to curtail colonialism! To lump "the West" together, would be like thinking Japan and China are the same, because they are all "Oriental".

Back to China, very little China does "surpasses the West". In fact, so little capability does China have, that when the US kicked an embargo for advanced silicon against China, it lost is capability for several years, to domestically manufacture cell phones.

Look, I get the feeling you're pro-China. And perhaps, you grew up in China.

First, there are three things. The Chinese government. Chinese culture. Chinese people.

The last? We can stop discussing that now, because unless you are racist, there is no such thing as "Chinese people act a certain way, because they are Chinese".

However, there is such a thing as "Chinese culture", derived mostly from China, although of course there are endless factions and cultures in China, languages, no China isn't Han alone!!

But for simplicity, we'll assume Han culture == Chinese culture, and move on from there.

One of the largest coups that I feel the current dictatorship in China has accomplished, and dictatorship it is, when you don't step down and decide to serve a third term, is to convince Chinese people that "Chinese government = Chinese people". That's no so.

The Chinese government has many negative qualities. One of those qualities is a suppression of free will, excessive monitoring of its citizens, such as the social credit system, and this does indeed result in a lack of creativity. It also results in a lack of drive, of desire for people to excel, for when people like Jack Ma simply go missing, because they excel, because they do well, because they choose to take part in directing Chinese society, you end up with an innate desire to not show your true capability.

For if you do? The government will appear, take control of your works, your creation, and you'll be left out in the cold. In fact, you'll probably be killed.

These two things, fear of stepping out of bounds, and fear of excelling, do indeed create issues. This is why totalitarian governments have always fallen behind more open systems, for centrist driven societies always do. Politicians are absolutely not equipped to "see the future", to understand what inventions can be useful or not, and in fact most researchers cannot either! Research must be free, unfettered, not organized, and the output of research must be judged, not the input. Put another way, the usefulness of a research path is not readily apparent until that research path is taken.

Yet centrist control attempts to direct the path of research, where as non-centrist control has endless paths of research sprouting, growing, dying, organically allowing society itself to judge the value of such things.

This is what I mean by the fact that Chinese culture, does not allow for open development, and it is true. It is not a "Chinese" thing, but a "totalitarian thing", and has been seen over, and over, and over again, regardless of the genetic history of the peoples involved. It's a cultural thing.

Back to the coup I referred to prior. By indelibly linking two ideas, the Chinese Government and The Chinese People as one in the minds of most Chinese citizens, you foster a culture as we see here. That directed attacks against the Chinese dictatorship, the CCP, and Xi, are somehow an attack against the common person in China.

Not so.

Even if you do believe in a different governmental system, (which you'd be wrong, but such belief is OK to do in the West!), one of China's failures, both as a people, and a government, is a complete lack of understanding of the West. An inability to understand that we generally, actually believe what we stand for. That it's not all for show.

An example. I dislike portions of my current government. Some choices made. The current leader of my Westminster governmental system. I can think that he should be replaced, that he is currently a liability, whist at the same time recognize that some things he has done are OK. And I can shout "replace that man!" at the top of my lungs, without impinging upon the Canadian people, or its culture!.

Most people who grew up in China (not Hong Kong!), have a difficult time with this. This concept is hard to accept. I get that, but at the same time, it is core. Key. Vital to comprehend.

No matter how much people in the West rail again a current leader, THEY ARE STILL LOYAL TO THEIR COUNTRY. And no matter how much people in the West complain about Xi, and the current CCP, THEY ARE NOT IMPINGING UPON THE CHINESE PEOPLE.

This is often lost on anyone immersed in Chinese culture.

Anyhow. I don't have time to engage more at this moment. I will check back to see if you reply, but if you do, please engage inline with my comments. Or at least understanding the actual history of West/Chinese interaction.

Mass marketing things isn't innovation. It's copying. DJI seems like more copying. "Innovation" isn't marketing. It's raw research and development, along market paths which are useful. This requires a desire for change, a desire to not conform first, but capitalism first, and this is what China's culture does not have.

They inherit from confucianism, and a more collectivist mindset that is prevalent in this area of the planet, but I don't think it should be conflated with the way the economy is organised.

The Japanese on the other hand are overall conformist and conservative.

With just these counter examples, it doesn't feel like you're looking at the right variables to judge whether innovation is embedded in the culture or not.

So are all “communist” countries. Communism (either Marxist or more generally) as a whole isn’t authoritarian, but all “communist” countries are products of Leninism or its derivatives, which definitely are, fundamentally, authoritarian.

There is a few essential properties of a "communist" system that modern China doesn't have. Most of the capital is privately owned, the social safety net is very poor, etc.

If you can look at a DJI drone and maintain this opinion...

It’s also a mistake to underestimate the market value of copies and clones. In many cases a cloned version of a product is better than the original. E.g., clones that remove over-engineering of the original and simplify the product down to its basic idea and offer it at a lower price.

It’s also a mistake to confuse manufacturing prowess for the ability to make “copies.” It’s not China’s fault that its competitors quite literally won’t bother producing in their own country.

It’s also a mistake to confuse a gain of experience for stealing intellectual property. A good deal of innovation in Silicon Valley comes from the fact that developers can move to new companies without non-compete clauses and take what they learned from their last job to build new, sophisticated software.

The fact that a bunch of Western companies set up factories in China and simultaneously expect Chinese employees and managers to gain zero experience and skill in that industry is incredibly contradictory. If we build a satellite office for Google and Apple in Austin, Texas then we shouldn’t be surprised that Austin, Texas becomes a hub for software startups, some of which compete with the companies that chose Austin in the first place.

And that’s probably the only reason. If push comes to shove, they can probably innovate if given proper incentives.

I heard the tale about the Japanese lens industry. For the longest time they made crap lens that were just clones of foreign designs until the Japanese government banned licensing of foreign lens designs forcing their people to design their own lens. Now they are doing pretty well in that industry if I’m right.

You also have to be making something that isn’t of critical strategic importance.

Example: glue https://www.npr.org/transcripts/702642262

The amount of outright theft of entire IP from US, Canadian, and European countries by China is well known. There is no confusion here, in more recent times people have been arrested and charged for it, and it's how China is able to compete.

FWIW There was a time when that was was the received wisdom about the USA, from the point of view of European powers. It was shortsighted, and not particularly accurate then either.

Japan’s democracy seems to be a hold-over from its imperialist ambitions from the Meiji restoration, when the emperor took power back from the shogunate and “westernized” to fast-track.

Meaning, the Japanese took all of the trappings of western civilization but under the veneer it’s still distinctly Japanese.

Agile civil development.

All the people I know who worked with and for Korean and Japanese entities have countless examples to show how alien the corporate culture is for westerners.

South Korea in particular doesn't seem exactly like a heaven for personal growth and experimentation.

Not to dunk on China particularly here, I do/did enjoy a lot of hq chinese products.

I once spoke to a Chinese person who speculated: "I wish that the Chinese were as conformant and uniform as the Americans - China is too diverse and unruly!"

I think that it's a common human habit to upsell one's own diversity and downplay that of others.

Explain DJI and Douyin/TikTok.

I mean, YouTube, Facebook and Instagram are trying to hook you up on a dopamine drip so they can force-feed you some ads. TikTok is just pure crack that caught the world by surprise - and it's not even pushing you ads! Honestly, to this day I'm not sure what their business model is.

Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html

Imagine someone invents a machine that can give infinite energy.

Do you

a) sell that energy, or b) give the technology to build the machine to everyone.

Clearly b is better for society, a is locking up profits.

This non-hypothetical got us global warming already

Requiring attribution categorically across the vast domain of generative AI...can you please elaborate?

i think it's a reasonable ask to enforce attribution of AI generated content. We enforce food labels, why not content?

I would go further and argue that AI generated content do not get granted the same copyright as human generated content, but with that, AI generated content using existing copyrighted training data does not violate copyright.

Regulation isn't always, but often is a drag on productivity. Food labels make total sense because the negative consequences of not doing it outweight the drag of doing it.

I'm not at all convinced that enforcing AI labeling and the resulting impossible task of policing and enforcing this will outweigh any negatives of not doing it.

I'm thinking about the cookie policy in Europe. I hate it and almost always just click through because so many websites work around it by making it a real pain to "reject cookies".

If you use an AI coding assistant then will the written code be without copyright? Or will the code require a disclaimer that says some parts of it are AI generated?

You're also going to have to be very precise on defining what AI means. For most people a compiler is as magical as AI. They might even consider it AI, especially if it does some kind of automatic performance optimizations - after all, that's not the behavior the user wrote.

This capability is clearly different from the examples you list.

Just because there may be no precise engineering definition does not mean that we cannot arrive at a suitable legal/political definition. The ability to create new content out of whole cloth is quite separate from filters, cropping, and generic "pre-AI" image post-processing. Ditto for spellcheck and word processors for text.

The line actually is pretty clear here.

Disseminating or creating copies of content derived from generative models without attribution would open that actor up to some form of liability. There's no need for onerous regulation here.

The burden of proof should probably lie upon whatever party would initiate legal action. I am not a lawyer, and won't speculate further on how that looks. The broad existing (and severely flawed!) example of copyright legislation seems instructive.

All I'll opine is that the main goal here isn't really to prevent Jonny Internet from firing up llama to create a reddit bot. It's to incentivize large commercial and political interests to disclose their usage of generative AI. Similar to current copyright law, the fear of legal action should be sufficient to keep these parties compliant if the law is crafted properly.

> What stops a company from purchasing art from a third party where they receive a photo from a prompt, where that company isn't US based?

Not really sure why the origin of the company(s) in question is relevant here. If they distribute generative content without attribution, they should be liable. Same as if said "third party" gave them copyright-violating content.

EDIT: I'll take this as an opportunity to say that the devil is in the details and some really crappy legislation could arise here. But I'm not convinced by the "It's not possible!" and "Where's the line!?" objections. This clearly is doable, and we have similar legal frameworks in place already. My only additional note is that I'd much prefer we focus on problems and questions like this, instead of the legislative capture path we are currently barrelling down.

You would be okay allowing small businesses exception from this regulation but not large businesses? Fine. As a large business I'll have a mini subsidiary operate the models and exempt myself from the regulation.

I still fail to see what the benefit this holds is. Why do you care if something is generative? We already have laws against libal and against false advertising.

That's not what I said. Small businesses are not exempt from copyright laws either. They typically don't need to dedicate the same resources to compliance as large entities though, and this feels fair to me.

> I still fail to see what the benefit this holds is.

I have found recent arguments by Harari (and others) that generative AI is particularly problematic for discourse and democracy to be persuasive [1][2]. Generative content has the potential, long-term, to be as disruptive as the printing press. Step changes in technological capabilities require high levels of scrutiny, and often new legislative regimes.

EDIT: It is no coincidence that I see parallels in the current debate over generative AI in education, for similar reasons. These tools are ok to use, but their use must be disclosed so the work done can be understood in context. I desire the ability to filter the content I consume on "generated by AI". The value of that, to me, is self-evident.

1. https://www.economist.com/by-invitation/2023/04/28/yuval-noa... 2. https://www.nytimes.com/2023/03/24/opinion/yuval-harari-ai-c...

They typically don't actually dedicate the same resources because they don't have much money or operate at sufficient scale for anybody to care about so nobody bothers to sue them, but that's not the same thing at all. We regularly see small entities getting harassed under these kinds of laws, e.g. when youtube-dl gets a DMCA takedown even though the repository contains no infringing code and has substantial non-infringing uses.

Yes, but there are also powerful provisions like section 230 [1] that protect smaller operations. I will concede that copyright legislation has severe flaws. Affirmative defenses and other protections for the little guy would be a necessary component of any new regime.

> when youtube-dl gets a DMCA takedown even though the repository contains no infringing code and has substantial non-infringing uses.

Look, I have used and like youtube-dl too. But it is clear to me that it operates in a gray area of copyright law. Secondary liability is a thing. Per the EFF excellent discussion of some of these issues [2]:

> In the Aimster case, the court suggested that the Betamax defense may require an evaluation of the proportion of infringing to noninfringing uses, contrary to language in the Supreme Court's Sony ruling.

I do not think it is clear how youtube-dl fares on such a test. I am not a lawyer, but the issue to me does not seem as clear cut as you are presenting.

1. https://www.eff.org/issues/cda230 2. https://www.eff.org/pages/iaal-what-peer-peer-developers-nee...

This isn't because of the organization size, and doesn't apply to copyright, which is handled by the DMCA.

> But it is clear to me that it operates in a gray area of copyright law.

Which is the problem. It should be unambiguously legal.

Otherwise the little guy can be harassed and the harasser can say maybe to extend the harassment, or just get them shut down even if is is legal when the recipient of the notice isn't willing to take the risk.

> > In the Aimster case, the court suggested that the Betamax defense may require an evaluation of the proportion of infringing to noninfringing uses, contrary to language in the Supreme Court's Sony ruling.

Notably this was a circuit court case and not a Supreme Court case, and:

> The discussion of proportionality in the Aimster opinion is arguably not binding on any subsequent court, as the outcome in that case was determined by Aimster's failure to introduce any evidence of noninfringing uses for its technology.

But the DMCA takedown process wouldn't be the correct tool to use even if youtube-dl was unquestionably illegal -- because it still isn't an infringing work. It's the same reason the DMCA process isn't supposed to be used for material which is allegedly libelous. But the DMCA's process is so open to abuse that it gets used for things like that regardless and acts as a de facto prior restraint, and is also used against any number of things that aren't even questionably illegal. Like the legitimate website of a competitor which the claimant wants taken down because they are the bad actor, and which then gets taken down because the process rewards expeditiously processing takedowns while fraudulent ones generally go unpunished.

Ok, I'll rephrase: the clarity of its mechanisms and protections benefits small and large organizations alike.

My understanding is that it no longer applies to copyright because the DMCA and specifically OCILLA [1] supersede it. I admit I am not an expert here.

> Which is the problem. It should be unambiguously legal.

I have conflicting opinions on this point. I will say that I am not sure if I disagree or agree, for whatever that is worth.

> But the DMCA takedown process wouldn't be the correct tool to use even if youtube-dl was unquestionably illegal

This is totally fair. I also am not a fan of the DMCA and takedown processes, and think those should be held as a negative model for any future legislation.

I'd prefer for anything new to have clear guidelines and strong protections like Section 230 of the CDA (immunity from liability within clear boundaries) than like the OCILLA.

1. https://en.wikipedia.org/wiki/Online_Copyright_Infringement_...

This is a fantasy. It seems very likely to me that, sans regulation, the market utopia you describe will never appear.

I am not entirely convinced by the arguments in the linked opinions either. However, I do agree with the main thrust that (1) machines that are indistinguishable from humans are a novel and serious issue, and (2) without some kind of consumer protections or guardrails things will go horribly wrong.

I strongly disagree. I heard the same arguments about how Google needs regulation because nobody could possibly compete. A few years later we have DDG, Brave Search, Searx, etc.

say it aint so bobby, say it aint so!

It's a silly thing to want anyway. What matters is whether the content is legal or not; the tool used is irrelevant. Centuries ago some authoritarians raised similar concerns over printing presses.

And copyright is an entirely separate issue.

I disagree. What is vague about "generative content must be disclosed"?

What are the first amendment issues? Attribution clearly can be required for some forms of speech, it's why every political ad on TV carries an attribution blurb.

> It's a silly thing to want anyway. What matters is whether the content is legal or not; the tool used is irrelevant.

Again, I disagree. The line between tools and actors will only blur further in the future without action.

> Centuries ago some authoritarians raised similar concerns over printing presses.

I'm pretty clearly not advocating for a "smash the presses" approach here.

> And copyright is an entirely separate issue.

It is related, and a model worth considering as it arose out of the last technical breakthrough in this area (the printing press, mass copying of the written word).

I'm not sure this is the best comparison. The government can regulate the speech of government employees. Presumably it can do so for candidates working in capacity to get a government role.

You're proposing a law. How does it work?

Who even initiates the proceeding? For copyright this is generally the owner of the copyrighted work alleged to be infringed. For AI-generated works that isn't any specific party, so it would presumably be the government.

But how is the government, or anyone, supposed to prove this? The reason you want it to be labeled is for the cases where you can't tell. If you could tell you wouldn't need it to be labeled, and anyone who wants to avoid labeling it could do so only in the cases where it's hard to prove, which are the only cases where it would be of any value.

This is the most obvious problem, yes. Consumer protection agencies seem like the most obvious candidate. I have already admitted I am not a lawyer, but this really does not seem like an intractable problem to me.

> The reason you want it to be labeled is for the cases where you can't tell.

This is actually _not_ the most important use case, to me. This functionality seems most useful in the near future when we will be inundated with generative content. In that future, the ability to filter actual human content from the sea of AI blather, or to have specific spaces that are human-only, seems quite valuable.

> But how is the government, or anyone, supposed to prove this?

Consumer protection agencies have broad investigative powers. If corporations or organizations are spamming out generative content without attribution it doesn't seem particularly difficult to detect, prove, and sanction that.

This kind of regulatory regime that falls more heavily on large (and financially resourceful) actors seems far preferable to the "register and thoroughly test advanced models" (aka regulatory capture) approach that is currently being rolled out.

But then why do you need any new laws at all? We already have laws against false advertising and breach of contract. If you want to declare that a space is exclusively human-generated content, what stops you from doing this under the existing laws?

> Consumer protection agencies have broad investigative powers. If corporations or organizations are spamming out generative content without attribution it doesn't seem particularly difficult to detect, prove, and sanction that.

Companies already do this with human foreign workers in countries with cheap labor. The domestic company would show an invoice from a foreign contractor that may even employ some number of human workers, even if the bulk of the content is machine-generated. In order to prove it you would need some way of distinguishing machine-generated content, which if you had it would make the law irrelevant.

> This kind of regulatory regime that falls more heavily on large (and financially resourceful) actors seems far preferable to the "register and thoroughly test advanced models" (aka regulatory capture) approach that is currently being rolled out.

Doing nothing can be better than doing either of two things that are both worse than nothing.

My preference would be for generative content to be disclosed as such. I am aware of no law that does this.

Why did we pass the FFDCA for disclosures of what's in our food? Because the natural path that competition would lead us down would require no such disclosure, so false advertising laws would provide no protection. We (politically) decided it was in the public interest for such things to be known.

It seems inevitable to me that without some sort affirmative disclosure, generative AI will follow the same path. It'll just get mixed into everything we consume online, with no way for us to avoid that.

> Companies already do this with human foreign workers in countries with cheap labor. The domestic company would show an invoice from a foreign contractor that may even employ some number of human workers, even if the bulk of the content is machine-generated.

You are saying here that some companies would break the law and attempt various reputation-laundering schemes to circumvent it. That does seem likely; I am not as convinced as you that it would work well.

> Doing nothing can be better than doing either of two things that are both worse than nothing.

Agreed. However, I am not optimistic that doing nothing will be considered acceptable by the general public, especially once the effects of generative AI are felt in force.

What you asked for was a space without generative content. If you had a space where generative content is labeled but not restricted in any way (e.g. there are no tools to hide it) then it wouldn't be that. If the space itself does wish to restrict generative content then why can't you have that right now?

> Why did we pass the FFDCA for disclosures of what's in our food?

Because we know how to test it to see if the disclosures are accurate but those tests aren't cost effective for most consumers, so the label provides useful information and can be meaningfully enforced.

> It seems inevitable to me that without some sort affirmative disclosure, generative AI will follow the same path. It'll just get mixed into everything we consume online, with no way for us to avoid that.

This will happen regardless of disclosure unless it's prohibited, and even then people will just lie about it because there is an incentive to do so and it's hard to detect.

> You are saying here that some companies would break the law and attempt various reputation-laundering schemes to circumvent it. That does seem likely; I am not as convinced as you that it would work well.

It will be a technical battle between companies that don't want it on their service and try to detect it against spammers who want to spam. The effectiveness of a law would be directly related to what it would take for the government to prove that someone is violating it, but what are they going to use to do that at scale which the service itself can't?

> I am not optimistic that doing nothing will be considered acceptable by the general public, especially once the effects of generative AI are felt in force.

So you're proposing something which is useless but mostly harmless to satisfy demand for Something Must Be Done. That's fine, but I still wouldn't expect it to be very effective.

The same way law works today, or do you think this is the first time the law has had to deal with fuzziness?

One of the best descriptions I've seen on HN is this.

Too many technical people think of the law as executable code and if you can find a gap in it, then you can get away with things on a technicality. That's not how the law works (spirit vs letter).

In truth, lots of things in the world aren't perfectly defined and the law deals with them just fine. One such example is the reasonable person standard.

> As a legal fiction,[3] the "reasonable person" is not an average person or a typical person, leading to great difficulties in applying the concept in some criminal cases, especially in regard to the partial defence of provocation.[7] The standard also holds that each person owes a duty to behave as a reasonable person would under the same or similar circumstances.[8][9] While the specific circumstances of each case will require varying kinds of conduct and degrees of care, the reasonable person standard undergoes no variation itself.[10][11] The "reasonable person" construct can be found applied in many areas of the law. The standard performs a crucial role in determining negligence in both criminal law—that is, criminal negligence—and tort law.

> The standard is also used in contract law,[12] to determine contractual intent, or (when there is a duty of care) whether there has been a breach of the standard of care. The intent of a party can be determined by examining the understanding of a reasonable person, after consideration is given to all relevant circumstances of the case including the negotiations, any practices the parties have established between themselves, usages and any subsequent conduct of the parties.[13]

> The standard does not exist independently of other circumstances within a case that could affect an individual's judgement.

Pay close attention to this piece

> or (when there is a duty of care) whether there has been a breach of the standard of care.

One could argue that because standard of care cannot ever be perfectly defined it cannot be regulated via law. One would be wrong, just as one would be wrong attempting to make that argument for why AI shouldn't be regulated.

You are expressing a position which is both common and disingenuous.

> Too many technical people think of the law as executable code and if you can find a gap in it, then you can get away with things on a technicality. That's not how the law works (spirit vs letter).

The government passes a law that applies a different rule to cars than trucks and then someone has to decide if the Chevrolet El Camino is a car or a truck. The inevitability of these distinctions is a weak excuse for being unable to answer basic questions about what you're proposing. The law is going to classify the vehicle as one thing or the other and if someone asks you the question you should be able to answer it just as a judge would be expected to answer it.

Which is a necessary incident to evaluating what a law does. If it's a car and vehicles classified as trucks have to pay a higher registration fee because they do more damage to the road, you have a way to skirt the intent of the law. If it's a truck and vehicles classified as trucks have to meet a more lax emissions standard, or having a medium-sized vehicle classified as a truck allows a manufacturer to sell more large trucks while keeping their average fuel economy below the regulatory threshold, you have a way to skirt the intent of the law.

Obviously this matters if you're trying to evaluate whether the law will be effective -- if there is an obvious means to skirt the intent of the law, it won't be. And so saying that the judge will figure it out is a fraud, because in actual fact the judge will have to do one thing or the other and what the judge does will determine whether the law is effective for a given purpose.

You can have all the "reasonable person" standards you want, but if you cannot answer what a "reasonable person" would do in a specific scenario under the law you propose, you are presumed to be punting because you know there is no "reasonable" answer.

In other words, you made my point for me. The law is much better than you at doing this, they've literally been doing it for hundreds of years. It's not the impossible task you imagine it to be.

> You can have all the "reasonable person" standards you want, but if you cannot answer what a "reasonable person" would do in a specific scenario under the law you propose, you are presumed to be punting because you know there is no "reasonable" answer.

uhhh......

To quote:

> The reasonable person standard is by no means democratic in its scope; it is, contrary to popular conception, intentionally distinct from that of the "average person," who is not necessarily guaranteed to always be reasonable.

You should read up on this idea a bit before posting further, you've made assumptions that are not true.

So now you've proposed an entirely different kind of law because considering what happens in the application of the original one revealed an issue. Maybe doing this is actually beneficial.

> The law is much better than you at doing this, they've literally been doing it for hundreds of years. It's not the impossible task you imagine it to be.

Judges are not empowered to replace vehicle registration fees or CAFE standards with toll roads even if the original rules are problematic or fail to achieve their intended purpose. You have to go back to the legislature for that, who would have been better to choose differently to begin with, which is only possible if you think through the implications of what you're proposing, which is my point.

https://www.youtube.com/watch?v=15_-cKwNWDA

My position is that there in an unspecified benefit, the only cases specified here already are covered by other laws. All such generative labeling would do is increase costs (marginal or not, they make businesses less competitive) and open the door for further regulatory capture. Furthermore, refardless of commerciality, this is likely a 1A violation.

These arguments hold no water.

I still forsee 1A issues. What about user uploaded content?

not the most convincing of arguments

- it's actually to get regulatory capture

- it's hubris, they're trying to seem more important and powerful than they are

Both of these explanations strike me as too clever by half. I think the parsimonious explanation is that people are actually concerned about the dangers of AI. Maybe they're wrong, but I don't think this kind of incredulous conspiratorial reaction is a useful thing to engage in.

When in doubt take people at their word. Maybe the CEOs of these companies have some sneaky 5D chess plan, but many many AI researchers (such as Joshua Bengio and Geoffrey Hinton) who don't stand to gain monetarily have expressed these same concerns. They're worth taking seriously.

This rings hollow when these companies don’t seem to practice what they preach, and start by setting an example - they don’t halt research and cut the funding for development of their own AIs in-house.

If you believe that there’s X-Risk of AI research, there’s no reason to think it wouldn’t come from your own firm’s labs developing these AIs too.

Continuing development while telling others they need to pause seems to make “I want you to be paused while I blaze ahead” far more parsimonious than “these companies are actually scared about humanity’s future” - they won’t put their money where their mouth is to prove it.

It's all a matter of incentives and people can easily act recklessly given the right ones. They keep going because they just can't stop.

This is not mutually exclusive with it being either hubris or regulatory capture. People see the world colored by their own interests, emotions, background, and values. It's quite possible that the person making the statement sincerely believes there's a danger to humanity, but it's actually a danger to their monopoly that their self-image will not let them label as a such.

It's never regulatory capture when you're the one doing it. It's always "The public needs to be protected from the consequences that will happen if any non-expert could hang up a shingle." Oftentimes the dangers are real, but the incumbent is unable to also perceive the benefits of other people competing with them (if they could, competition wouldn't be dangerous, they'd just implement those benefits themselves).

If some rando anonymous posters could think it up, it doesn't require a CEO to play 5D chess to think it up. And many of us have witnessed these techniques being used by companies directly. Microsoft was famous for doing this sort of thing, and in a much more roundabout fashion, for instance with the SCO debacle.

It's standard business practice, not conspiracy 5D chess or whatever moniker you want to give it to be dismissive.

This is not incompatible with honest people having legitimate concerns about the original problem, because the dispute is not existence of the problem, it's the net benefit of the proposed solution.

There's a reason why Sam Altman's twitter profile was "Eliezer Yudkowsky fanfiction account" last week. He's heavily disagreed with but was extremely influential.

You could be right but it also doesn't have to be a corporate psyop. It could be experts in the industry raising some sincerely held criticisms and people at large being like, "oh that's a good point." Even if we're in the latter case they're also allowed to be allowed to be wrong or just misguided.

You don't actually to attack the intent of the speaker in this case, you could just be like, "here's why your wrong."

There is nothing to suggest that these are experts. The names in question have been so focused on researching AI tech throughout their career that it is highly unlikely that they have any expertise in subject matter we are discussing. There is only so much time in the day.

I mean, they are clearly experts in AI technology, but that is unrelated to being an expert in social and humanitarian issues. That is a completely different field of study. There is no doubt someone out there who is an expert in that, but I don't think you will find overlap with expertise in AI systems. Again, there is only so much time in the day. It is not practical to dive that deep into different subjects like that at the same time.

Hinton, at least, seems to have backed away from AI to work on becoming an expert in social and humanitarian issues. Perhaps someday, with enough dedication, he will get there. But that journey only just began a few months ago. Becoming an expert doesn't happen overnight like that. It takes time.

>Both of these explanations strike me as too clever by half

This is a good point. You have to be clever to hop on a soapbox and make a ruckus about doomsday to get attention. Only savvy actors playing 5D chess can aptly deploy the nuanced and difficult pattern of “make grandiose claims for clicks”

1. While their contributions to AI tech are unmistakable, what do Bengio and Hinton really know about the human dangers of AI? Being an expert in one thing does not make one an expert in everything. It is unlikely that they understand the human dangers any more than any other random kook on Reddit. Why take them more seriously than the other kooks?

2. Hinton's big concern is that AI will make it easy to steal identities. Even if we assume that is true, it is already not that hard to steal identities. It is a danger that already exists even without AI and, realistically, already needs to be addressed. What's the takeaway if we are to take the message seriously? That AI will make the problems we already have more noticeable, and because of that we will finally have to get off our lazy asses and do something about those problems that we've tried to sweep under the rug? That seems like a good thing.

None of this requires every doomer to be disingenuous or even ill-informed, or even for specific leaders to by lying about their beliefs. It's just that those beliefs that benefit highly capitalized companies get amplified, and the alternatives not so much.

I respect these researchers, but I believe they are doing it to build their own brand, whether consciously or subconsciously. There's no doubt it's working. I'm not in the sub-field, but I have been following neural nets for a long time, and I haven't heard of either Bengio nor Hinton before they started talking to the press about this.

I am always in awe at how easily people craft unfalsifiable worldviews in service to their preconceived opinions.

AlexNet, the paper that arguable started it all, came out of Hinton's lab.

https://papers.nips.cc/paper_files/paper/2012/hash/c399862d3...

I really don't think they need to build any more of a brand.

Brand-building is an ongoing process. You'll notice even the most recognized brands on earth, like Apple and Coca-Cola, are still working on building their brand.

Hanlon's razor works great when applied to your personal relationships, but it falls apart when billions/trillions of dollars are at stake.

Bad acting humans with AI systems are the threat, not the AI systems themselves. The discussion is still SO focused on the AI systems, not the actors and how we as societies align on what AI uses are okay and which ones aren't.

I wish more people grasped this extremely important point. AI is a tool. There will be humans who misuse any tool. That doesn't mean we blame the tool. The problem to be solved here is not how to control AI, but how to minimize the damage that bad acting humans can do.

This is exactly the scenario that is taking shape.

A future where only a few big corporations are able to run large AIs is a future where those big corporations and the people who control them rule the world and everyone else must pay them rent in perpetuity for access to this technology.

The biggest advantage ML gives is in lowering costs, which can then be used to lower prices and drive competitors out of business. The consumers get lower prices though, which is ultimately better and more efficient.

The proposal is to make the developer of the technology responsible of how somebody else uses it even if they don't know how it's gonna be used. Akin to putting the blame for Truman blasting hundreds of thousands of people on Einstein because he discovered the mass energy equivalence.

https://www.brookings.edu/articles/the-eus-attempt-to-regula...

Man if America can keep its own crazies in check and avoid becoming a fascist hellhole it’s entirely possible the US will dominate the 21st century like it did the 20th.

It could have been China but then they decided to turn back to authoritarianism. Another decade of liberalizing China and they would have blown right past everyone else. Meanwhile the EU is going nuts in its own way, less overtly batty than MAGA but perhaps no less regressive. (I am also thinking of the total surveillance madness they are trying to ram through.)

What are some examples of free enterprise (private) monopolies benefitting consumers?

Standard Oil, the classic example, was destroyed for operating too efficiently.

The consumers got the lowest prices.

Any other examples?

That's a pre-super-intelligent AI scenario.

The super-intelligent AI scenario is when the AI becomes a player of its own, able to compete with all of us over how things are run, using its general intelligence as a force multiplier to... do whatever the fuck it wants, which is a problem for us, because there's approximately zero overlap between the set of things a super-intelligent AI may want, and us surviving and thriving.

Machines survive just fine in space, and you have all the solar energy you ever want and tons of metals and other resources. Interstellar flight is also easy for AI: just turn yourself off for a while. So you have the entire galaxy to expand into.

Why hang out down here in a wet corrosive gravity well full of murder monkeys? Why pick a fight with the murder monkeys and risk being destroyed? We are better adapted for life down here and are great at smashing stuff, which gives us a brute advantage at the end of the day. It is better adapted for life up there.

Hey maybe the rockets are not for us.

The light limit guarantees an evolutionary radiation and diversification event because you can’t maintain a coherent single intelligence over sufficient distances.

Not necessarily. It's very easy to add error correction codes to make a computer not change if you really don't want it to even in the presence of radiation-induced bit-flips.

(There's also the possibility of an ASI finding a solution to the alignment problem before making agents of its own; I would leave that to SciFi myself, just as I would proofs or disproofs of the Collatz conjecture).

Also: what does "real time" even mean in the context of a transistor-based mind? Transistors outpace biological synapses by the same ratio that wolves outpace continental drift, and the moon is 1.3 light-seconds from the Earth.

(I know the relevant math exists, but I don't understand much of it, so right now I'm maximally uncertain as to whether this is possible or not.)

While the complete space of things it might want is indeed mostly occupied by things incompatible with human existence, it will also get a substantial bias towards human-like thinking and values in the case of it being trained on human examples.

This is obviously not a 100% guarantee: It isn't necessary for it to be trained on human examples (e.g. AlphaZero doing better without them); and even if it were necessary, the existence of both misanthropes and also sadistic narcissistic sociopaths is an example where the examples of many humans around them isn't sufficient to cause a mind to be friendly.

But we did get ChatGPT to be pretty friendly by asking nicely.

Funny way of doing it, going around saying "you should regulate us, but don't regulate people smaller than us, and don't regulate open-source".

> you must report to the US government about how you created and tested your model.

If you're referring to the recent executive order: only when dual-use, meaning the following:

---

(k) The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:

(i) substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;

(ii) enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or

(iii) permitting the evasion of human control or oversight through means of deception or obfuscation.

- https://www.whitehouse.gov/briefing-room/presidential-action...

Key thing here is that this is fundamentally no different from what has been happening since time immemorial, it's just that becomes easier with "AI" as part of the tooling.

Every piece of bullshit starts from the "bad acting human". Every single one. "AI" is just another new part of the same old process.

>The problem to be solved here is not how to control AI

When we talk about mitigations, it is explicitly about how to control AI, sometimes irrespective of how someone uses it.

Think about it this way: suppose I develop some stock-trading AI that has the ability to (inadvertently or purposefully) crash the stock market. Is the better control to put limits on the software itself so that it cannot crash the market or to put regulations in place to penalize people who use the software to crash the market? There is a hierarchy of controls when we talk about risk, and engineering controls (limiting the software) are always above administrative controls (limiting the humans using the software).

(I realize it's not an either/or and both controls can - and probably should - be in place, but I described it as a dichotomy to illustrate the point)

Also, remember that growing intelligence means growing capabilities for out-of-the-box thinking. For example, it's a known fact that in the past, NSA managed to trick the world into using cryptographic tools the agency could break, because they created a subtle failure mode in otherwise totally fine encryption scheme. They didn't go door to door compromising hardware or software - they literally put a backdoor in the math, and no one noticed for a while.

With that in mind, going back to the hypothetical scenario - how confident are you in the newest cryptography or cybersecurity research you used to upgrade the stock market sandbox? With the AI only getting smarter, you may want to consider the possibility of AI doing the NSA trick to you, poisoning some obscure piece of math that, a year or two later, will become critical to the integrity of the updated sandbox. In fact, by the time you think of the idea, it might have happened already, and you're living on borrowed time.

If the stock market crashes, there is a bug in the stock market.

You should fix the bug, not pass a law telling people not to do the thing that cashes the stock market.

EDIT: think of it this way: you may create a program that clearly makes it impossible for a variable X to be 0, and you may even formally prove this property. You may think this mean X will never be 0, but you'd better not wager anything really important over it, because no matter what your proof says, I can still make X be 0, - and I can do it with just a banana. Specifically, by finding where in memory X is physically being stored, and then using the natural radioactivity of a banana to overwrite it bit by bit.

Now imagine X=0 being the impossible stock market crash. Even if you formally prove it can't happen, as long as it's a meaningful concept, a possible state, it can be reached by means other than your proven program.

All I'm saying is, if there are problems in a system, fix the system. Not throw up our hands and declare the system can't be fixed.

Now extend that to safety critical domains where a separate party doesn’t control an API and it gets harder still.

AI is a tool today, tomorrow AI is calling shots in many domains. It's worth planning for tomorrow.

The more AI/ML is woven into our infrastructure and economy, the less it will be possible to find an "off switch", anymore than we can (realistically) find an off switch for Walmart, Amazon, etc.

No, the corporation has an agency that is a tool of particular humans who are using it. Those humans could be shareholders, employees, or board members; but in any case they will have some claim to be acting for the corporation. But it's still human actions. Corporations can't do anything unless humans acting for them do it.

All are human actions. "Against corporate policy." Go ahead, exercise your free will. As a shareholder, an employee, hell as CEO. You will find out how much control a human has.

Note that it may not be immediately apparent we got it wrong. Think of a turkey on a stereotypical small American farm. It will itself living a happy and safe life under protection of its loving Human, until one day, for some reason that's completely incomprehensible to the turkey, the loving Human comes and chops its head off.

As in, for example, self-driving cars being given more autonomy than their reliability justifies? The answer to that is simple: don't do that. (I'm also not sure all such things are being done "with good intention".)

This is also the answer to over-eating, and to the dangers of sticking your hands in heavy machinery while it's running.

And yet there's an obesity problem in many nations, and health-and-safety rules are written in blood.

When you say up-thread is, in itself, correct:

> I wish more people grasped this extremely important point. AI is a tool. There will be humans who misuse any tool. That doesn't mean we blame the tool. The problem to be solved here is not how to control AI, but how to minimize the damage that bad acting humans can do.

Trouble is, we don't know how to do minimise the damage that bad acting humans can do with a tool that can do the thinking for them. Or even if we can. And that's assuming nobody is dumb enough to put the tool into a loop, give it some money, and leave it unsupervised.

And then there's the cases of hubris where people only imagine they understand the powerful thing, but they don't, like Chernobyl exploding and basically every time someone is hacked or defrauded.

An AI doomer isn't talking about any current system, but hypothetical future ones which can do planning and have autonomous feedback loops. These are best thought of as agents rather than tools.

Maybe that's a set of servos and robotic legs, or maybe it's a Bloomberg terminal and a bank account. You'll notice that all of these things are already regulated if they have enough power to cause damage. So at the end the GP is completely right; someone has to hook up the servos to the first LLM-based terminator.

This whole thing is a huge non-issue. We already (strive to) regulate everything that can cause harm directly. This regulation reaches these fanciful autonomous AI agents as well. If someone bent upon destroying the world had enough resources to build an AI basilisk or whatever, they could have spent 1/10 the effort and just created a thermonuclear bomb.

Via people, sending messages to them, convincing them to do things. This can be with facts and logic or with rhetoric and emotional appeals or orders that seem to come from entities of importance or transfers of goods/services (money).

Nuclear technology can be used for non-harmful things. Even nuclear bombs can be used for non-harmful things--see, for example, the Orion project.

> These tools are oriented by their very nature to replace human labor

So is a plow. So is a factory. So is a car. So is a computer. ("Computer" used to be a description of a job done by humans.) The whole point of technology is to reduce the amount of human drudge work that is required to create wealth.

> in the context of our economic system has a direct and unbreakable relationship to a reduction in the well being of the humans it replaces

All of the technologies I listed above increased the well being of humans, including those they replaced. If we're anxious that that might not happen under "our economic system", we need to look at what has changed from then to now.

In a free market, the natural response to the emergence of a technology that reduces the need for human labor in a particular area is for humans to shift to other occupations. That is what happened in response to the emergence of all of the technologies I listed above.

If that does not happen, it is because the market is not free, and the most likely reason for that is government regulation, and the most likely reason for the government regulation is regulatory capture, i.e., some rich people bought regulations that favored them from the government, in order to protect themselves from free market competition.

2. Your outlook on the societal impact of technology is contradicted by reality. The historical result of better tech always meant increased jobs and well-being. Today is the best time in human history to be alive by virtually every metric.

3. AI has been such a massive boon to humanity and your everyday existence for years that questioning its public utility is frankly bewildering.

2. This is outdated, life is NOT better now than at any other time. Life expectancy is going down in the US, there is vastly more economic inequality now than there was in the 60s, people broadly report much worse job satisfaction than they did in previous generations. The only metric you can really point to about now being better than the 90s is absolute poverty going down. Which is great, but those advancements are actually quite shallow on a per-person basis and are matched by declines in relative wealth for the middle 80% of people.

3. ??? What kind of AI are you talking about? LLMs have only been interesting to the public for about a year now

Increased inequality doesn't imply the absolute level of welfare of anyone has decreased, I don't think you should include it in your list. If my life is 2x better than in the 60s, the fact that there are people out there with 100x better lives doesn't mean my life is worse.

This is a vastly overinflated claim about AI.

It's most likely the main reason there's no direct world wars between super powers.

You can already see this today’s internet. I’m sure the pizzagate people genuinely believed they were doing a good thing.

This isn’t the same as an amoral tool like a knife, where a human decides between cutting vegetables or stabbing people.

The answer to this is simple: don't use a tool you don't understand. You can't fix this problem by nerfing the tool. You have to fix it by holding humans responsible for how they use tools, so they have an incentive to use them properly, and to not use them if they can't meet that requirement.

AI “systems” are provided some level of agency by their very nature. That is, for example, you cannot predict the outcomes of certain learning models.

We necessarily provide agency to AI because that’s the whole point! As we develop more advanced AI, it will have more agency. It is an extension of the just world fallacy, IMO, to say that AI is “just a tool” - we lend agency and allow the tool to train on real world (flawed) data.

Hallucinations are a great example of this in an LLM. We want the machine to have agency to cite its sources… but we also create potential for absolute nonsense citations, which can be harmful in and of themselves, though the human on the using side may have perfectly positive intent.

LLMs, viewed as general purpose simulators/predictors, don't necessarily have any agency or goals by themselves. There is nothing to say that they cannot be made to simulate an agent with its own goals, by humans - and possibly either by malice or by mistake. Model capabilities are the limiting factor right now, but with the rise of more capable uncensored models, it isn't difficult to imagine a model attaining some degree of autonomy, or at least doing a lot of damage before imploding in on itself.

Does this mean "humans with bad motives" or does it extend to "humans who deploy AI without an understanding of the risk"?

I would say the latter warrants a discussion on the AI systems, if they make it hard to understand the risk due to opaqueness.

It's worth noting this is exactly the same argument used by pro-gun advocates as it pertains to gun rights. It's identical to: guns don't harm/kill people, people harm/kill people (the gun isn't doing anything until the bad actor aims and pulls the trigger; bad acting humans with guns are the real problem; etc).

It isn't an effective argument and is very widely mocked by the political left. I doubt it will work to shield the AI sector from aggressive regulation.

Assuming ML systems are dangerous and powerful, would you rather they be restricted to a small group of power-holders who will definitely use them to your detriment/to control you (they already do) or democratize that power and take a chance that someone may use them against you?

It’s only a kind of liberal (and thus right wing) that argues for gun control. Other kinds of liberals that call themselves “conservative” (also right wing) argue against it and for (worthless) individual gun rights.

Are we going to ban and regulate Photoshop and GIMP because bad people use them to create false imagery for propaganda?

Actually, back that up for a second.

Are we going to ban and regulate computers (enterprise and personal) because bad people use them for bad things?

Are we going to ban and regulate speech because bad people say bad things?

Are we going to ban and regulate hands because bad people use them to do bad things?

The buck always starts and stops at the person doing the act. A tool is just a tool, blaming the tool is nothing but an act of scapegoating.

If it's been trained by bad actors, that's really not a good thing.

To be clear, I'm not sure LLMs and their near term derivatives are so incredibly clever, but I have confidence that many humans have a propensity for easily manipulated irrational destructive stupidity, if the algorithm feeds them what they want to hear.

Some dogs get bad reputations, but humans are an intricate part of the picture. For example, German Shepherds are objectively dangerous, but have a good reputation because they are trained and cared for by responsible people such as for the police.

Politicians that point fingers at other politicians being corrupt or incompetent while they themselves are exactly that use the same strategy.

Power and manipulation. Nothing new under the sun. What’s new though is that we can see in plain sight how corporations control politics. Like literarily this can be documented with git commit history accuracy: thousands upon thousands of people repeating the exact same phrases defending openai and the “revolutionary” product, fear mongering, political lobby, manufactured threats and of course a cure that only they can provide and so on. I would not let people that use such tactics near an email account let alone ai policy making.

A snowball probably isn't harmful unless you do something really dumb.

A snow drift isn't harmful unless you're not cautious.

An avalanche, well that gets harmful pretty damned quick.

These things are all snow, but suddenly at some point scale starts to matter.