This begs the question, does the G-factor (IQ) curve have diminishing returns and are humans meaningfully on the diminishing returns side of the curve?
It's not at all obvious that a super AI would have an intellect incomprehensible to a human the same way a human is to a chimpanzee.
Or another way to phrase it, it's not at all obvious that an AI can exist that would be incomprehensible to a very smart human, it may however reason much faster than such a human.
A very good question, and a very hard question to answer. From a point of view of a horse, a super horse is just a faster horse. A 2000cc super-bike is incomprehensible to a horse, because it exists in a different category of speed.
I think similarly, a super AI's intelligence will be a completely different type of intelligence than what we have. For the lack of better word, it exists in a different dimension.
For example, things that are blackbox to us will be completely comprehensible to that super AI. Or it can solve problems that we might have considered impossible to solve.
>I think similarly, a super AI's intelligence will be a completely different type of intelligence than what we have. For the lack of better word, it exists in a different dimension.
This is nonsense, magical thinking. It's possible to model reasoning formally; it's called "logic". A system of deductions built upon some axioms. Any logical reasoning, no matter how complex, can be expressed in such a system, and can be understood by anyone else given enough time.
Humans emulate (or simulate) logic, but it's not our natural state.
Computers are perfect logicians by default, but AFAIK no logic compact enough to be human-comprehensible has been enough to see, hear, or read. At least, not reliably so.
Logic gates are combined to form binary numbers, which are used to label symbols and approximate reals, upon which calculus is approximated in toy models of neurons, which are combined and trained and eventually learn to add numbers and then puts the wrong number of hands onto the third arm of the human it was tasked with drawing.
We do know that AI's have at least some advantages. For example they're more accurate and orders of magnitude faster at floating point calculations. They'll also have communication interfaces with other (sub)systems that are way more precise and fast than anything we could manage. And then there's the perfect memory.
I imagine such advantages also come with at least a lower time bound on solving many classes of problems and presumably that would be experienced as an incomprehensibly smart intelligence. I imagine it'd feel like chess computers in most areas of life in that the AIs actions would feel impossibly perfect at all turns leaving us far behind in attempts to compete
The G-factor in IQ, even disregarding the discussion about how useful it is or isn't for humans, is not the only factor when considering non-humans.
An AI mind which can learn and intuit as well as an IQ 130 human (2σ, the tests become unreliable above that), but also comes with the speed difference between synapses and transistors (roughly the same as the difference between jogging and continental drift), has a chance to become expert at every subject.
Most of us have enough difficulty truly comprehending the domains of other single human experts; a human-upload with that much breadth of expertise will be incomprehensible by default even if they speak your language.
> This begs the question, does the G-factor (IQ) curve have diminishing returns and are humans meaningfully on the diminishing returns side of the curve?
It definitely has "diminishing returns to acquiring resources". There are people many IQ points higher than Musk but none of them are anywhere near as wealthy as him, and it's not clear that Musk would be richer if he was smarter.
A ~1sd difference in average g at the national level is the difference between not having stable electricity, and being an uber-rich technological wonderland. That suggests that even if the curve is bending, it's not a very hard bend.
Ultimately it boils down to lack of competition + enormous overheads due to regulation, legal & political risk and uncertainty.
The accidents were spectacular, even though they resulted in so very few deaths. For most people nuclear is effectively witchcraft, and selling fear is effective. The HBO series were so absurd yet so loved.
Or just clone whatever API the official Reddit mobile client is using. As long as it's offered for free to the official app, there's no technical way to stop another app from using the same API. The best they could do is bundle some private keys in the official app, but ultimately anything on the client can be reverse engineered and cloned by another app.
The only solution Reddit has to that is complaining to Apple, who can reject the third party app from the App Store. There's precedent for this with things like "unofficial" Pokémon Go clients. Apple is usually happy to remove them. But I'm not sure it's ever gone to trial - it would certainly be interesting, given case law around APIs like with Oracle v Google, or LinkedIn v HiQ.
What would you do with the g measurement to get your location? Can g deltas be matched to sea floor contour maps? Otherwise it seems like subs would have to be following g-mapped routes.
With a highly detailed map, it becomes a statistical fitting problem, to locate yourself based on an estimate of position history using intertidal measurements and your time history of gravitational measurements, starting from an initial position estimate. I would imagine this could be quite accurate.
A typical sub drifts by only a few tens of meters per day - obviously this builds up . The point of the BEC gravity map is to reduce this inaccuracy an order of magnitude, thereby extending the subsurface range of the subs.
By having a base map of the gravity of the ocean floor. Picture a contour map, but for values of g.
Same as if you were navigating based on a physical contour map of the ocean floor, except that getting an accurate depth like that in deep waters isn’t possible, and in shallow waters requires sonar.
I'd assume features below the sea floor have a much bigger impact than surface features. Large iron deposits, volcanic activity, natural gas deposits, whatever the LLSVPs are, etc. But you could use ships to correlate gravity measurements with GPS locations and make an accurate map that way. You don't need to map the entire ocean, just enough locations to allow subs to occasionally recalibrate their position.
This sort of move has no downsides for the incumbents. Either they succeed and achieve regulatory capture or they poison the well sufficiently that further regulation will not be feasible.
Ultimately, the reward for attaining an AGI agent is so high, that no matter the penalty someone will attempt it, and someone will eventually succeed. And that likelihood will ensure everyone will want to attempt it.
For torrenting, port forwarding is only marginally important - for torrents which have very few peers and you can connect to none of them.
It's also risky because mullvad certainly has records of forwarded ports and can out you if they receive a properly worded subpoena. There is also a chance those records would be present in their backups even after you deleted the forwarded ports.
I have a separate command for port forwarded torrent client and only use it when absolutely necessary, which is almost never.
How is that relevant here? mullvad has to keep track of who to forward the port to, any NAT ports are going to be ephemeral and conducted through an encrypted tunnel.
The reason why regulation is going to fail is simple. AGI is the supreme bet, the Gamble to Resurrection. Faced with death years to decades from now or a chance at amortality. Which would you choose?
Would you be willing to bet on p(doom) whatever it might be when p(amortality | AGI) >>> p(amortality) during your lifetime? I personally am willing to place such a bet, and I would hazard a guess that decision makers don't come to their positions without similar sentiments.
I think this is what our tech overlords are investing all the money for. They're getting older quite quickly and probably feel like immortality is just around the corner if they keep syncing money into AI research and keep taking risks, they will cheat death.
I compare it to the the great pyramids. The pyramids were arguably another technological marvel also built on the back of slaves (just like modern tech now) and probably a lot of great inventions and innovations were made to facilitate their creation. To people of the time, the pyramids must've been absolutely incredible marvels to behold.
But all those Pharo's are dead, just like everyone else. I think the current tech overlords will die like everyone else too.
Nature already has a way for people to live on, it's called having children and dying. It's a great system and no one ever gets bored. However, the ego of some people is so strong and isn't satisfied with this, so on we go pouring money and resources into the search for the holy grail no matter what the risks are to everything else.
To some people death and the end of the universe are equivalent events. Others seek refuge in natural fallacies.
The difference between this moment and all others, is that amorality is actually achievable. If you can get an AGI and drown it in compute, through sheer brute-force the secrets of biology will be unraveled. The task is not to attain amortality immediately, merely gain more life faster than you lose it.
The only way what your saying is true is if we could create people with zero links to their parents DNA or create completely rewrite all DNA so it's novel (we can't).
This doesn't mean you own your children or anything like that, no one is suggesting their not independent, but you do go on living through your children in one way or another.
Technically untrue, approximately 7.31 percent of all humans who have ever lived are currently alive (i.e. have avoided death (so far)).
Also your argument is just disingenuous. Nonexistence may not be painful, but the concept of nonexistence can be quite disturbing for the currently existing (even apart from the experience of dying.)
Do you think any of those 7.31 percent of all humans will avoid death?
And no, it’s not disingenuous. If someone offered you the choice of immortality (never ever dying under any circumstances) or continuing with natural human existence (being guaranteed to die at some point), which would you take?
I think kids born recently have higher chance to avoid death - in next 70 years a lot of can change. And even if in that time frame we won't be able to make people immortal maybe we can extend lifespan to 150 years with still healthy body? Then they would buy themself extra 80 years for another health innovations in race to immortality.
And immortality is not like you cannot die - it's just you wouldnt die becasue od aging. You could also kill yourself if you get bored after living 200 years. Today most people don't commit suicide because they got bored. Most people don't give up after having cancer. Average lifespan is 80 years and people know it and kind of accept it but would you be happy to have lifespan as dog or cat less than 20 years?
I think many people would want to have a choice to live 200+ years and in good health.
Don’t see one that says, “Unending life unless you want to kill yourself after 200 years.”
> Average lifespan is 80 years and people know it and kind of accept it but would you be happy to have lifespan as dog or cat less than 20 years?
If the average lifespan was 20 years then it wouldn’t bother me any more than a lifespan of 80 years bothers me today. Which it is to say that it doesn’t bother me at all.
Not sure if some one definition picked without context is good example. Immortal doesn't have to mean eternal life.
In green mythology we have few examples of supposedly immortal gods that died.
Elves in Tolkien were immortal but could die.
On the other hand in Christianity Jesus was a mortal God that died and go ressurected and is now believed to be immortal. Buddhist believe in reincarnation.
The universe is supposed to die eventually at one point. In this way even if AGI could be immortal or keep resurrecting itself it would eventually die when universe dies.
Over the pandemic and thereafter my family has had a lot of deaths in it. So, I've been forced to think about it a lot, to sit with others over it, and to just deal with a lot of the mundane parts of it too.
Death is horrible. It makes no sense. When people say they have had a 'loss', they're not kidding. You feel like that person should still be there, but like a kid at Disneyland, they are lost to you and you're searching for them. I'm dealing with a fair few relatives that just are not processing all these losses. Grief is a strange, personal, and unique thing too. It manifests personally and yet stereotypically for every individual.
That said, I think Death is the way.
The reasoning is complex and long. So, I'll try to sum it up for a simple comment, and I'll do that poorly, sorry.
The big reason stem from this article on Wikipedia. I think it's one of the best there is on the whole site:
Say you were truly immortal. That timeline would be your future, as far as we currently understand physics. You get to spend a lot of time between universes if the last entry is to be believed. In fact, they don't even bother with giving the units. Nanoseconds or gigayears are pretty much the same st those timescales. Our time here on Earth is essentially as brief as the entire non-black-hole era to an immortal like that. Purgatory in the black void is more like what such an immortal would experience.
Or say you get to relive your life when you die. Poof, you're reborn to your folks and have all your memories again somehow. Repeat forever. You're doomed to live and die the same life, like 'Groundhog Day', but for ~80 years long and not just a day. Another purgatory after enough lives, I'd guess. Sisyphean.
Heaven, hell. An afterlife is our best hope. Somewhere we can't possibly understand with our minds right now. A total lack of understanding of the life hereafter is the only path where you retain something of you. Where you can grow and change, time can continue in, I dunno, 7 dimensions. I've not a clue, and I think that's great. If I did, right now I think you'd just end up in a form of purgatory given enough time.
But an endless dreamless night is just fine too. In no other way except an afterlife that I cannot possibly understand do we get to have 'happily ever after'. I think everyone would take that Socratic apology given enough time and I think they'd be right to do so.
I dunno, been sitting on this a while, and it's late for me and, again, sorry that this is a brief and jumbled comment.
If you are in the tech industry in the US this is actually a good thing, EU is trying very hard to remain and become even more uncompetitive.
Already, the EU is mostly a breeding ground for talent that is then extracted by the US. The more hostile and bureaucratic they become, the grater the pressure for the talent to leave.
It may appear that what the EU is doing is being hostile to US companies - and some individual US companies will indeed suffer. But the actual effect is to incentivize these companies to extract talent out of the EU as an insurance plan in the event of a pull out.
Given that native English speakers have a leg up in HR processes over here, I agree that everything that makes Europe a better living space is a nice thing for Americans who want out and largely can get out.
I work for a European tech company, and we have a noticeable uptick of American applicants over the years. Common (as in: almost every time) interview questions I answer are "Is it safe there? I work in [say, Portland] and we had just had gunshots again across the street" and "Can my kids ride the subway alone?". Colleagues who made the leap tend to tell me they are "glad they got out".
There’s some truth in what you wrote, but certainly this brand of “competitiveness” (meaning tragedy of the privacy commons) should not be considered a worthy goal to compete on.
These things are typically bureaucratic security theater. Terrorists can sideload. The decision makers just care to be seen doing something, whether it's useful or not is not their problem.
You severely underestimate the people in power, and the people who stand behind them.
Such bans are deliberate attacks targeting (parts of) the general population. Terrorists have nothing to do with this.
"Hahaha those stupid bureaucrats, they have no idea what they are doing" is a popular meme, but it couldn't be further from the truth. They do know what they are doing, and it's working. If you believe it isn't, that's only because you misunderstand what the real goal is.
Absolutely. I know from personal experience that much of what remains of independent journalism in India, as well as some NGO work (long under government scrutiny) is conducted through Signal and Element.
Perhaps I should add that I do not wish to undermine the terrorism situation in India, which is very much real. Regardless, there is a paradox because at the same time the associated terrorism laws and hacking operations have been misused to target opposition, journalists, scientists, activists, and many others.
The amount of people who get caught on insecure communication, including some famous tech CEOs, is staggering. Most people, criminal or not, are completely security/tech illiterate. Changing the default has a huge impact. Webs in these investigations tend to be wide so it's not just about literal criminals, if one suspect leaks info to anyone that may be enough for authorities to catch on, it's a statistical game basically and the harder you make it to not mess up the higher your chance to catch someone.
Gentle reminder that the US intelligence leaker was caught because he distributed the material on a gaming discord on an account with user info that was one Google search away from his father's instagram and steam profile. And that guy was US National Guard tech support staff.
I disagree. This may or may not be effective, but it's not theatre.
Criminals, terrorists, dissidents, politicians, even intelligence people sometimes... they're users. Just because they can sideload, or otherwise use more sophisticated methods doesn't mean they will. The margins are wide.
Imo, same thing is happening in India as is happening in most places with an active security-intelligence service. They receive valuable intelligence from PMs. Then they fret about potentially losing access or want to increase access.
If you read HN/Reddit/Twitter, you'd get the impression that all this stuff is fake. It's not. Intelligence agencies in 2023 are all about these sources, and jelously guard them. Once they have a source, they're not giving it up willingly.
Certainly a ban like that, if it can be made at least somewhat effective, will prevent a lot of uses of encrypted messaging by less sophisticated users. I'm afraid it's not going to be effective against the stated targets, the well-prepared terrorists sponsored by a neighboring state. Those will be trained to sideload, or whatever else it takes.
Even "somewhat effective" is better than no ban. Given the state of terrorism in some parts, this is a crucial tool. For example in the aftermath recent terrorist bombing in Pulwama, clamping down on Internet had helped.
When this stuff started happening in Russia qlmost a decade ago, everybody was joking about bureaucrats instead of taking measures. Fast forward a few years, Internet access is not unlike cheese with holes in it, and a great number of people had to invest in VPN access.
Disagree - this is unsubstantiated and worthy of an Elon Musk style of how he stripped that BBC reporter down. If they were the targets, Journalists, dissidents would have completely dissappeared over time. They are active and vociferous.
They do deep packet inspection on the telco end and drop packets to the blocked servers. Sure there might a way to circumvent, especially if there’s a p2p app, but even those packets may be detected and dropped.
I don't understand what people are doing to be able to assert that a streaming service is more convenient than piracy. For me, piracy is more convenient than streaming by a significant margin.
Visit one of the very public torrent sites such as rarbg, search or browse for something of interest, copy magnet URI, switch to terminal window, type aria2c "[magnet uri]". Just minutes later have the file in working directory. Now you can play it with MPV and have instantaneous seeks and no garbage web players.
Sounds like a lot more steps than clicking play on a web app. You also have to do that for single episode of a new tv show as it gets released as opposed to clicking play on a web app
It's not at all obvious that a super AI would have an intellect incomprehensible to a human the same way a human is to a chimpanzee.
Or another way to phrase it, it's not at all obvious that an AI can exist that would be incomprehensible to a very smart human, it may however reason much faster than such a human.