> Our belief is that once we can no longer come up with quantifiable problems that are "feasible for humans and hard for AI" then we effectively have AGI.
I don’t think that follows. Just because people fail to create ARC-AGI problems that are difficult for an AI to solve, doesn’t mean that said AI can just be plugged into a humanoid robot and it will now reliably cook dinner, order a pizza and drive to pick it up, take a bus to downtown to busk on the street and take the money back home, etc.
ARC-AGI is an interesting benchmark, but it’s extremely presumptive to think that these types of tests are going to demonstrate AGI.
That’s precisely what I meant in my comment by “these types of tests.” People are eventually going to have some sort of standard for what they consider AGI. But that doesn’t mean the current benchmarks are useful for this task at all, and saying that the benchmarks could be completely different in the future only underscores this.
How are any of these a useful path to asking an AI to cook dinner?
We already know many tasks that most humans can do relatively easily, yet most people don’t expect AI to be able to do them for years to come (for instance, L5 self-driving). ARC-AGI appears to be going in the opposite direction - can these models pass tests that are difficult for the average person to pass.
These benchmarks are interesting in that they show increasing capabilities of the models. But they seem to be far less useful at determining AGI than the simple benchmarks we’ve had all along (can these models do everyday tasks that a human can do?).
The task you mention require intelligence but also a robot body with a lot of physical dexterity suited to a designed-for-humanoids world. That seems like an additional requirement on top of intelligence? Maybe we do not want an AGI definition to include that?
There are humans who cannot perform these tasks, at least without assistive/adapted systems such as a wheelchair and accessible bus.
> at least without assistive/adapted systems such as a wheelchair and accessible bus.
Which is precisely what the robotic body I mentioned would be.
You're talking about humans who have the mental capacity to do these things, but who don't control a body capable of doing them. That's the exact opposite of an AI that controls a body capable of doing these things, but lacks the mental capacity to do them.
I read that has “humans can perform these task, at least with…”
Put the computer in a wheelchair of his choice and let him try to catch the bus. How would you compare program and human reasoning abilities, but disregarding human ability to interact with the outside world?
Edit: Arc-AGI itself is only approachable by visually and manually valid humans, others needs assistive devices.
> Of course, I’m self-aware enough to recognize that it might be because my writing is terrible or because I’m covering topics no one cares about.
I've hard similar issues with you, and actually think the opposite to be the case. When I was trying to build an audience, I actually found that it was the low effort nonsense that would get the most traction. At a certain point I was attempting to try to get followers by making a lot of those, and then trying to mix some quality posts in, and had some success. But I started asking myself, to what end? What kind of community am I building that's only interested in low quality junk?
And one thing I noticed about Blogging and Twitter is that they're extremely cliquish. From what I've seen, most people would rather interact with popular Bloggers/Twitter users that they hate or say are idiots than they would with users who have a low follow count/seldom read blog. Sure, there are ways you can juice your follower count so that you're large enough that the big guys will think you're worth enough to pay attention to. But again - what's the point? When you see the complete vapidity of many of these supposed thought leaders, is it really worth it? So what, you can get into the daily Twitter slap fights that they seem to love so much?
I mostly wish their were more places were thoughtful people to find and chat with each other, without having to get involved with petty blog/Twitter vanity games.
That's a good point, and I completely agree. It's not that I was seeking fame or attention, but rather I wanted to put out ideas and find like-minded people (in my case, saas founders/"indiehackers") to talk shop with. Even if it were just a couple people each time, that would be plenty, but I still felt that was beyond reach.
Even this HN reply thread alone is more engagement than I would normally get on social media ;). I'm not saying this to complain, I'm okay with it and it's just the nature of the internet with all its noise. But to get back on topic, I can totally see why people don't choose to maintain a blog as much anymore.
Maybe I'm just not a Real Artist, but I don't understand this focus on "engagement" and "visibility" for casual writers and other online publishers. Assuming they are not doing it for revenue, where their income depends on huge readership, why are they so concerned with how many readers they are getting? When I share some source code on GitHub, I don't care in the slightest whether anybody or nobody uses it. It doesn't really affect me. Same for comments on HN. I get no benefit if 10,000 people read a comment vs. 100.
Whenever you talk about blogging vs. more popular platforms, someone always chimes in with this "but I get so many more eyeballs on Twitter!" and I legitimately don't understand why that matters.
Sure, if you are doing it for a living and your income scales with the number of readers, then yea, of course, it's obvious why you want "engagement."
>Assuming they are not doing it for revenue, where their income depends on huge readership, why are they so concerned with how many readers they are getting?
Are you including indirect revenue in your assumption? There's lots of hopeful incentives that don't immedialy lead to money:
- you get a reputation and get professional gigs or invitations or whatnot. Fame, in a word.
- You get a reputation and that makes it easier to validate your next pitch for some dream idea you have. For better or worse, saying "a lot of people like this" is very effective pitch material.
- you want to meet other like minded people and organically network. These can lead to future opportunities you would have never considered.
- You have some larger societal mission, and that requires society. If you have some altruistic goal of say, teaching everyone to code (to pick a cliche idea), then you need people to participate to realize your goal. Something like Khan Academy still needs to advertise itself.
Your view only really applies to people who want to do Art for themselves. But we are still a social species, we have a natural urge to share our creations, for profit or not.
One of the things is wanting to be part of the discourse. For instance, this has happened to me several times - big players are talking about a particular topic. I dig through the primary sources, and see that many of the assumptions people are making about it are wrong. I try to bring it up, but - where? Blog/Tweet about it, and with no audience you're yelling into the void. Sometimes I try contacting the big players, but like I said, it's a pretty cliquish environment, and if you're a nobody you get ignored. Another option is to spit out a lot of garbage dopamine hits to build up a big enough audience to the point where someone might pay attention to your good points.
In the end I just gave up, because I realized the state of discourse in these spaces is terrible. It's a shame, though, because there are a lot of small, overlooked voices that do similar things, diving through primary source material and data and uncovering very important stuff that's gotten ignored. Occasionally, I've seen these people break stories that eventually get the attention of the national media - but it's hard, and this usually only happens for the really huge stories.
Meanwhile, the big players in these spaces are usually intellectually incurious and busy churning out vapid engagement bait.
>I mostly wish their were more places were thoughtful people to find and chat with each other, without having to get involved with petty blog/Twitter vanity games.
Sadly a lot of that moved to Discord. So, you can't really search it up. You have to simply follow a rabbit trail of people and forums until you find that community. It's tough.
But maybe that semi-elusiveness is a feature. You let anyone in and you may start to fall into that Eternal September where the low quality vapidness wins out
That's very much the impression I get. I've never seen pinyin used in Chinese writing, and the Chinese friends I've met have said they've never seen it either (they said they'd probably just look up the character or write a homonym instead, but even then it's pretty rare that it comes to that).
That's not to say it's never done, but it feels like an outlier. As if a friend found a word too hard to understand and drew a picture instead, and then the author wrote an article about how spelling is so difficult that it leads English speakers to draw words instead of writing them.
But the thing that struck me the most was just how confused people were when I asked them about it. It just didn't seem to be anything that was an actual issue for them.
Removing the added information would make it much more difficult to parse, though. Paragraphs don't exist in oral English - or spaces between words, quotation marks, capitalization, etc. - but we still find it much more easy to read properly formatted text than improperly formatted text.
Just because people are able to understand strict phonetic transcriptions, doesn't mean it's a good way to convey information (which is why almost no language relies on just strict phonetic transcriptions).
From what I've heard people say (and what I've seen), the most common way to handle it is to simply write another character that sounds the same. In other words, the characters can be used as phonetic elements when it's needed. It looks weird (in the same way that spelling words phonetically in English can look word), but it's doable.
That's for situations where they had to write something by hand but didn't have their phone with them to check (otherwise they can just spend a second to look at the character), which isn't a common occurrence.
The shrimp example is kind of strange. Like you said, it's an extremely common character, and not a difficult one either. But beyond that, if you look at it he got the radical, 虫, correct. The phonetic element, 下, is a fundamental character that I doubt anyone forgets to write.
It just seems like such an odd outlier example. Like talking about a friend that spells "been" as "bin." I'm sure it could happen, but it's not indicative of a broader trend.
The story was reported by Victor Mair, though, who is extremely opposed to using characters and often exaggerates the issues with them.
Personally, I've seen a lot of Chinese people's written notes, and I don't think I've ever seen them resort to pinyin, even among people that didn't go to college. I just asked a few Chinese friends about this, and they told me they never resort to pinyin either.
It would result in a pretty severe loss of fidelity.
You may think it’s not needed, because that information isn’t available in spoken Chinese. The same is true for written English - putting spaces between words, dividing texts into paragraphs, capitalizing them, differentiating between different pauses (a comma, period, semicolon, etc. all signifying what kind of pause something its), quotation marks, parenthesis, etc. - none of this is available in our spoken language, and we’re still able to understand it. In theory, we could get rid of them all and understand what’s being written. In practice, most people would find the result to be an incomprehensible mess.
The same goes for Chinese. Written languages, for the most part, are more than a simple transcription of spoken sounds.
> In practice, most people would find the result to be an incomprehensible mess.
Unless Chinese is somehow unique among all human languages, this isn't true. Chinese would be just as intelligible if written in a phonetic script (like Pinyin) as it is when written using the characters.
Now, it would be an incredibly shocking transition for Chinese people who have already spent their entire lives writing with characters. However, after the transition to Pinyin, especially for young people who wouldn't ever learn the characters, written Chinese would still be perfectly understandable.
That being said, I don't favor replacing the characters, because the transition would be extremely difficult and because the characters are very culturally important to China. They've been in use for a good 3000 years, and people are very attached to them. Phonetic scripts are technically superior, but the cultural and practical arguments for sticking with the characters are still stronger.
> Unless Chinese is somehow unique among all human languages, this isn't true.
I was talking about English in that paragraph:
> The same is true for written English - putting spaces between words, dividing texts into paragraphs, capitalizing them, differentiating between different pauses (a comma, period, semicolon, etc. all signifying what kind of pause something its), quotation marks, parenthesis, etc. - none of this is available in our spoken language, and we’re still able to understand it. In theory, we could get rid of them all and understand what’s being written. In practice, most people would find the result to be an incomprehensible mess.
> So you were talking about both English and Chinese in that sentence.
I was talking about English in the sentence you quoted. In the next paragraph, I said that Chinese was the same as English in this regard. That's why I couldn't (and still can't) understand your comment.
You're saying it isn't true that removing those parts of English would mean "most people would find the result to be an incomprehensible mess" unless Chinese is unique? Chinese has absolutely no connection to written English becoming a mess after removing those elements of written English.
Or are you objecting to the paragraph after the one you quoted, where I say the same thing that happens in English is true for Chinese? "Unless Chinese is somehow unique among all human languages, this isn't true" that Chinese would be like English? That doesn't make any sense to me unless you misread my initial comment to mean the complete opposite of what it was saying.
It's very clear what you meant, and I don't know why you're going in circles like this.
You very clearly wrote that Chinese would become an incomprehensible mess if written in Pinyin.
You first stated that there would be a severe loss in fidelity in switching to Pinyin. Then you gave an analogy showing how removing various non-phonetic elements of written English would make it an incomprehensible mess. Immediately after that, you said that the same applies for Chinese.
I'm objecting to your argument that Chinese would be an incomprehensible mess if written alphabetically.
No, I'm genuinely confused by your claim that in order for Chinese to be similar to English in this manner, it would be "somehow unique among all human languages." These are contradictory ideas. That's why I was asking for clarity.
> I'm objecting to your argument that Chinese would be an incomprehensible mess if written alphabetically.
That's fine, but it runs directly counter to your initial comment. If a phonetic transcription would make Chinese just as easy to understand as it is written now, it would be quite different from English, and almost every other written language, all of which include non-phonetic elements in order to facilitate reading.
I'm not sure what's confusing you. You laid out your initial argument clearly. I laid out my response clearly.
Now, you're obsessing over some pretty obvious misinterpretations of what I've written, and you're ignoring the argument you yourself initially made.
> If a phonetic transcription would make Chinese just as easy to understand as it is written now, it would be quite different from English, and almost every other written language, all of which include non-phonetic elements in order to facilitate reading
Pinyin, the phonetic transcription of Standard Chinese, is written with spaces and punctuation. You're going on about something that doesn't exist.
Even less of a difference. There's no equivalent character, so it's clearly 烹 written with an extra stroke. It's more like the difference between writing "deceive" and decieve."
> I guess you could write down 景茶 (jing3cha2) and rely on the phonetics, or use a different word if you know one, but it's still wrong on a level that "choclit" just isn't.
If a non-native did this in their own way would likely look wrong, but Chinese natives do occasionally use phonics to write or to substitute some characters with others.
The fact that 嚔/sneeze is usually the go to example means it ends up becoming the exception that proves the rule. Most other characters are much more easily remembered.
If you learn languages you quickly notice that knowing the 99% most frequent words still means that you need to look up a word every paragraph or two and that you have trouble expressing yourself. To write Chinese you need to know several thousand characters, forgetting just a few dozen can be quite annoying when you try to write nontrivial texts.
Being able to write a character by hand, being able to type it up, and being able to read it are all different things. I doubt many Chinese would be thrown off from reading or typing 打喷嚏.
I actually did a deep dive into the issue of unfamiliar characters coming up when reading, and how people handle them. I won't go into all the details, but the general takeaway is:
1. Unfamiliar characters can actually be quite rare or quite common depending on the material you're reading.
2. It's not much of an issue for people either way.
I don’t think that follows. Just because people fail to create ARC-AGI problems that are difficult for an AI to solve, doesn’t mean that said AI can just be plugged into a humanoid robot and it will now reliably cook dinner, order a pizza and drive to pick it up, take a bus to downtown to busk on the street and take the money back home, etc.
ARC-AGI is an interesting benchmark, but it’s extremely presumptive to think that these types of tests are going to demonstrate AGI.