What matters is if there is a shared representation space across languages. If there is, you can then (theoretically, there might be a PhDs and a Nobel or two to be had :) separate underlying structure and the translation from underlying structure to language.
The latter - what they call the universal embedding inverter - is likely much more easily trainable. There's a good chance that certain structures are unique enough you can map them to underlying representation, and then lever that. But even if that's not viable, you can certainly run unsupervised training on raw material, and see if that same underlying "universal" structure pops out.
There's a lot of hope and conjecture in that last paragraph, but the whole point of the article is that maybe, just maybe, you don't need context to translate.
I don't know about a Lion, but I think Wittgenstein could have benefited from having a pet.
I train my cat and while I can't always understand her I think one of the most impressive features of the human mind is to be able to have such great understanding of others. We have theory of mind, joint attention, triadic awareness, and much more. My cat can understand me a bit but it's definitely asymmetric.
It's definitely not easy to understand other animals. As Wittgenstein suggests, their minds are alien to us. But we seem to be able to adapt. I'm much better at understanding my cat than my girlfriend (all the local street cats love me, and I teach many of them tricks) but I'm also nothing compared to experts I've seen.
Honestly, I think everyone studying AI could benefit by spending some more time studying animal cognition. While not like computer minds these are testable "alien minds" and can help us better understand the general nature of intelligence
Cats are domestic animals, and dogs are even more.
You probably didn't adapt to understanding cats as much as cats have adapted over millennia to be understood by humans. Working with and being understood by the dominant specie that is humans is a big evolutionary advantage.
Understanding a wild animal like a lion is a different story. There is a reason why most specialists will say that keeping wild animals as pets is a bad idea, they tend to be unpredictable, which, in other words, mean we don't understand them.
I agree with the logic, but I don't think it doesn't properly accounts for people who work very closely with wild animals.
You or I, yeah, probably not going to understand a lion pretty well. But someone who works at the zoo? A lion tamer? Someone studying lion cognition? Hell, people have figured out how to train hippos so that they can clean their teeth[0], and these are one of, if not the, most aggressive animals in the world. Humans have gotten impressively good at communicating with many different animals and training them. There's plenty of Steve Irwin types who have strong understandings about many creatures who would be quite alien to the rest of us. Which, that requires at least one side to have a strong understanding of the other's desires and how they perceive the world. But me? I have no doubt that hippo would murder me.
My point isn't so much about would we understand the lion, but rather could we. Wittgenstein implied we wouldn't be able to. I'm pointing to evidence that we, to at least some degree, can. How much we ultimately will be able to, is still unknown. But I certainly don't think it is an impossible task.
FWIW I don't think this is quite true for domestic cats - cross-species collaboration is very common in carnivores. A famous example is the coyote and badger, which will team up to hunt prairie dogs: the badger chases them out of the burrows, the coyote catches them outside, and they split the meat 70-30 in acknowledgement that the coyote is larger and needs more food. Recently the same has been observed with ocelots and possums. It seems like most mammals are able to at least somewhat understand the mood and intentions of other mammals - note that we share facial expressions and body language.
OTOH feral cats are known for being highly social compared to other cats, forming large semi-collaborative colonies. And adult cats have much more difficulty socializing to humans than adult dogs, even if they don't have trauma/etc. I suspect the real story of cat domestication goes both ways: an unusually gregarious subspecies of African wildcat started forming colonies near human settlements and forming cross-carnivore collaborations with the humans who lived there. This was also true for dogs - it likely started with unusually peaceful Siberian wolves - but I believe cats were more "accidental." Humans have been deliberately creating dog breeds since antiquity, but with a tiny number of exceptions cat breeds are modern. I doubt ancient humans ever "bred" cats like they did dogs, it seems closer to natural selection.
Fwiw, a lot of cat breeds are very old. Especially the ones coming out of northeast Africa (like Egypt).
But yes, both accidental domestication happens as well as non human cross species collaboration. Another famous example is with the cleaner fish and sharks. Animals also frequently collaborate with plants. Ants even have farms, both fungi and other insects
I think the response is generally you are communicating with your cat as an animal, as a mammal. Yes, communication is possible because we too are mammals, animals, etc.
But Lion is not just animal, it is not just mammal, it is something more. Something which I have no idea how we would communicate with.
No. I'm saying the areas you point out that you feel you could communicate, are ones in which you share a lived experience with the lion. You are both animals, you are both mammals. You get cold, hungry, thirsty, etc.
But lions, and us, are not just animals + mammals. Being a lion or a human means more. Ultimately, there is a uniquely human or lion element. Wittgenstein is saying we cannot communicate this.
It has multiple readings out of context, but I think in context one is much clearer than the other. Language is heavily overloaded, which is ironically directly relevant to the conversation.
Also, your comment still made me laugh. Women can be mysterious...
There is nothing really special about speech as a form of communication. All animals communicate with each other and with other animals. Informational density and, uhhhhh, cyclomatic complexity might be different between speech and a dance or a grunt or whatever.
I was referencing Wittgenstein's "If a lion could speak, we would not understand it." Wittgenstein believed (and I am strongly inclined to agree with him) that our ability to convey meaning through communication was intrinsically tied to (or, rather, sprang forth from) our physical, lived experiences.
Thus, to your point, assuming communication, because "there's nothing really special about speech", does that mean we would be able to understand a lion, if the lion could speak? Wittgenstein would say probably not. At least not initially and not until we had built shared lived experiences.
If we had a sufficiently large corpus of lion-speech we could build an LLM (Lion Language Model) that would “understand” as well as any model could.
Which isn’t saying much, it still couldn’t explain Lion Language to us, it could just generate statistically plausible examples or recognize examples.
To translate Lion speech you’d need to train a transformer on a parallel corpus of Lion to English, the existence of which would require that you already understand Lion.
Hmm I don't think we'd need a rosetta stone. In the same way LLMs associate via purely contextual usage the meaning of words, two separate data sets of lion and English, encoded into the same vector space, might pick up patterns of contextual usage at a high enough level to allow for mapping between the two languages.
For example, given thousands of English sentences with the word "sun", the vector embedding encodes the meaning. Assuming the lion word for "sun" is used in much the same context (near lion words for "hot", "heat", etc), it would likely end up in a similar spot near the English word for sun. And because of our shared context living in earth/being animals, I reckon many words likely will be used in similar contexts.
That's my guess though, note I don't know a ton about the internals of LLMs.
Someone more knowledgeable might chime in, but I don't think two corpuses can be mapped to the same vector space. Wouldn't each vector space be derived from its corpus?
It depends how you define the vector space but I'm inclined to agree.
The reason I think this is from evidence in human language. Spend time with any translator and they'll tell you that some things just don't really translate. The main concepts might, but there's subtleties and nuances that really change the feel. You probably notice this with friends who have a different native language than you.
Even same language same language communication is noisy. You even misunderstand your friends and partners, right? The people who have the greatest chance of understanding you. It's because the words you say don't convey all the things in your head. It's heavily compressed. Then the listener has to decompress from those lossy words. I mean you can go to any Internet forum and see this in action. That there's more than one way to interpret anything. Seems most internet fights start this way. So it's good to remember that there isn't an objective communication. We improperly encode as well as improperly decode. It's on us to try to find out what the speaker means, which may be very different from the words they say (take any story or song to see the more extreme versions of this. This feature is heavily used in art)
Really, that comes down to the idea of universal language[0]. I'm not a linguist (I'm an AI researcher), but my understanding is most people don't believe it exists and I buy the arguments. Hard to decouple due to shared origins and experiences.
Hmm I don't think a universal language is implied by being able to translate without a rosetta stone. I agree, I don't think there is such a thing as a universal language, per se, but I do wonder if there is a notion of a universal language at a certain level of abstraction.
But I think those ambiguous cases can still be understood/defined. You can describe how this one word in lion doesn't neatly map to a single word in English, and is used like a few different ways. Some of which we might not have a word for in English, in which case we would likely adopt the lion word.
Although note I do think I was wrong about embedding a multilingual corpus into a single space. The example I was thinking of was word2vec, and that appears to only work with one language. Although I did find some papers showing that you can unsupervised align between the two spaces, but don't know how successful that is, or how that would treat these ambiguous cases.
> I don't think a universal language is implied by being able to translate without a rosetta stone.
Depends what you mean. If you want a 1-to-1 translation then your languages need to be isomorphic. For lossy translation you still need some intersection within the embedding space. The intersection will determine how good you can translate. It isn't unreasonable to assume that there are some universal traits here as any being lives in this universe and we're all subject to these experiences at some level, right? But that could result in some very lossy translations that are effectively impossible to translate, right?
Another way you can think about it, though, is that language might not be dependent on experience. If it is completely divorced, we may be able to understand anyone regardless of experience. If it is mixed, then results can be mixed.
> The example I was thinking of was word2vec
Be careful with this. If you haven't actually gone deep into the math (more than 3Blue1Brown) you'll find some serious limitations to this. Play around with it and you'll experience these too. Distances in high dimensions are not well defined. There also aren't smooth embeddings here. You have a lot of similar problems to embedding methods like t-SNE. Certainly has uses but it is far too easy to draw the wrong conclusions from them. Unfortunately, both of these are often spoken about incorrectly (think as incorrect as most peoples understandings of things like Schrodinger's Cat or the Double Slit experiment, or really most of QM. There's some elements of truth but it's communicated through a game of telephone).
That's a very good point! I hadn't thought of that. And that makes sense, since the encoding of the word "sun" arises from its linguistic context, and there's no such shared context between the English word sun and any lion word in this imaginary multilingual corpus, so I don't think they'd go to the same point.
Apparently one thing you could do is train a word2vec on each corpus and then align them based on proximity/distances. Apparently this is called "unsupervised" alignment and there's a tool by Facebook called MUSE to do it. (TIL, Thanks ChatGPT!) https://github.com/facebookresearch/MUSE?tab=readme-ov-file
Although I wonder if there are better embedding approaches now as well. Word2Vec is what I've played around with from a few years ago, I'm sure it's ancient now!
Edit: that's what I get for posting before finishing the article! The whole point of their researh is to try to build such a mapping, ve2vec!
And even, assuming the existence of a Lion to English corpus, it would only give us Human word approximations. We experience how lossy that type of translation is already between Human->Human languages. Or sometimes between dialects within the same language.
Who knows, we don't really have good insight into how this information loss, or disparity grows. Is it linear? exponential? Presumably there is a threshold beyond which we simply have no ability to translate while retaining a meaningful amount of original meaning.
Would we know it when we tried to go over that threshold?
Sorry, I know I'm rambling. But it has always been regularly on my mind and it's easy for me to get on a roll. All this LLM stuff only kicked it all into overdrive.
> In broad terms, the Hypothesis claims that the limits of the language one speaks are the limits of the world one inhabits (also in Wittgenstein), that the
grammatical categories of that language define the ontological categories of the
word, and that combinatory potentials of that language delimit the complexity of
that world (this may be Jim Brown's addition to the complex Hypothesis.) The
test then is to see what changes happen in these areas when a person learns a
language with a new structure, are they broadened in ways that correspond to the
ways the structure of the new language differs from that of the old?
That seems a bit extreme, given that a lion also has a mammal brain. I'd expect it to also think in terms of distinct entities that can move around in the environment and possibly talk about things like "hunger" and "prey".
I'd expect incomprehensible language from something that is wildly different from us, e.g. sentient space crystals that eat radiation.
On the other hand, we still haven't figured out dolphin language (the most interesting guess was that they shout 3D images at each other).
Observation has proven enough to understand the meaning of animal calls. People proved they correctly identified, for example, an distressed animal call for assistance, by playing it to their peers in the wild. They go look for the distressed animal. Other calls don't provoke the same reaction.
Analogies are always possible. I believe in the philosophical context though, understanding the meaning of something is not possible through analogy alone.
Reminds me of the quote:
“But people have an unfortunate habit of assuming they understand the reality just because they understood the analogy. You dumb down brain surgery enough for a preschooler to think he understands it, the little tyke’s liable to grab a microwave scalpel and start cutting when no one’s looking.”
Hmm I'm not convinced we don't have a lot of shared experience. We live on the same planet. We both hunger, eat, and drink. We see the sun, the grass, the sky. We both have muscles that stretch and compress. We both sleep and yawn.
I mean who knows, maybe their perception of these shared experiences would be different enough to make communication difficult, but still, I think it's undeniably shared experience.
That's fair. To me, the point of Wittgenstein's lion thought experiment though was not necessarily to say that _any_ communication would be impossible. But to understand what it truly meant to be a lion, not just what it meant to be an animal. But we have no shared lion experiences nor does a lion have human experiences. So would we be able to have a human to lion communication even if we could both speak human speech?
I think that's the core question being asked and that's the one I have a hard time seeing how it'd work.
Hmm, I'm finding the premise a bit confusing, "understand what it truly meant to be a lion". I think that's quite different than having meaningful communication. One could make the same argument for "truly understanding" what it means to be someone else.
My thinking is that if something is capable of human-style speech, then we'd be able to communicate with them. We'd be able to talk about our shared experiences of the planet, and, if we're capable of human-style speech, likely also talk about more abstract concepts of what it means to be a human or lion. And potentially create new words for concepts that don't exist in each language.
I think the fact that human speech is capable of abstract concepts, not just concrete concepts, means that shared experience isn't necessary to have meaningful communication? It's a bit handwavy, depends a bit on how we're defining "understand" and "communicate".
> I think the fact that human speech is capable of abstract concepts, not just concrete concepts, means that shared experience isn't necessary to have meaningful communication?
I don't follow that line of reasoning. To me, in that example, you're still communicating with a human, who regardless of culture, or geographic location, still shares an immense amount of shared life experiences with you.
Or, they're not. For example, an intentionally extreme example, I bet we'd have a super hard time talking about homotopy type theory with a member of the amazon rain forest. Similarly, I'd bet they had their own abstract concepts that they would not be able to easily explain to us.
I would say there's a difference between abstract and complex. A complex topic would take a lot of time to communicate mainly because you have to go through all the prerequisites. By abstract I mean something like "communicate" or "loss" or "zero"! The primitives of complex thought.
And if we're saying the lion can speak human, then I think it follows that they're capable of this abstract thought, which is what I think is making the premise confusing for me. Maybe if I change my thinking and let's just say the lion is speaking... But if they're speaking a "language" that's capable of communicating concrete and abstract concepts, then that's a human-style language! And because we share many concrete concepts in our shared life experience, I think we would be able to communicate concrete concepts, and then use those as proxies to communicate abstract concepts and hence all concepts?
Does a lion not know what it's like to be hungry? These parts of the brain are ancient. There is clearly a sliding scale in most experiences here from amoeba to fly to lion to human. Would you like to communicate with a girl who drinks tapioca milk tea? Clearly your life experiences are different so what's the point? Obviously gets harder, that's why we are discussing the possibility of using technology to make it easier.
Obviously it's impossible to communicate even 90% of human experience with lions or people with mental disabilities. But if a translation model increases communication even 1%, brings everybody up to the level of a Kevin Richardson it's a huge win E.g. A pair of smart glasses that labeling the mood of the cat. Nobody cares about explaining why humans wear hats to a lion and of course no explanation is better than being a old human who has worn hats for a variety of reasons.
I think it's unlikely you could make a LLM that gives a lion knowledge via audio only, but very possibly other animals
>There is billions of human-written texts, grounded in shared experience that makes our AI good at language. We don't have that for a whale.
the world around us is a very large part of that shared experience. It is shared among humans, shared among whales, and shared among whales and humans as well.
If we could help gorillas or elephants (both highly intelligent) learn to name things and use symbols — in a form they can comprehend and create to express their will — enabling them to pass down their experiences and wisdom across generations, I believe they could quietly be as smart as we are.
Ps. I am excited about Google’s Gemma dolphin project (https://blog.google/technology/ai/dolphingemma/), but I would prefer if they chose elephants instead of dolphins as the subject, since we live on land, not in water. This way, more emphasis could be placed on core research, and immediate communication feedback would be possible.
I think it would be extremely surprising for Universal Grammar to be proven false. We have not had any shortage of opportunity to teach animals structured language, and yet we have nothing to show for it. It seems pretty likely that a key factor in our fitness as a species was having the right hardware for grammar.
> I think it would be extremely surprising for Universal Grammar to be proven false.
All you need to make it unsurprising you need to find an alternative hypothesis, that can explain observed facts.
How about this one. Humans got the ability to abstract deeper than others. While some animals could deal with concepts closely associated with real phenomena, and therefore they can use words to name things or maybe even actions, they cannot go further and use abstractions over abstractions. As the result they cannot deal with a recursive grammar.
Or another hypothesis. Humans got an ability to reflect on themselves, so they came to concepts like "motive (of action)", it made it possible for them to talk about actions, inventing words-categories, that categorize actions by supposed motive of action. But it is not just that, people started to feel that simple words do not describe the reality good enough, because a) actions have more than one motive, b) they started to see actions everywhere (the stone is lying flat? it is the action of the stone, that strictly speaking cannot act.). All this led them to invent complex grammars to describe the world where they live. Animals from other hand can't get it, because their world is much simplier. They don't see the stone "lying": it is a stone, it cannot do anything.
These hypotheses don't seem explaining everything, but I have just invented them. I didn't really tried to explain everything. But thinking about it, I'm in favor of the second one: all people are schizophrenics if you compare them with animals. They believe in things that are not exist, they see inanimate objects as animate, they are talking with themselves (inner dialog). It just happens that the particular kind of schizophrenia most people are struggling with is a condition that allows them to not tear apart all the connections with the Reality. It is more like borderline schizophrenia.
> While some animals could deal with concepts closely associated with real phenomena, and therefore they can use words to name things or maybe even actions, they cannot go further and use abstractions over abstractions. As the result they cannot deal with a recursive grammar.
I prefer to think that animals simply haven’t been lucky enough to invent a suitable naming system that could serve as the foundation of their civilization. If they were fortunate, they might bootstrap their own form of civilization through recursive or iterative divisive naming-—naming the act of naming itself. Naming is the foundation, and everything else naturally follows from there.
As Laozi said over 2,500 years ago in the Dao De Jing:
“The Tao that can be spoken is not the eternal Tao.
The name that can be named is not the eternal name.
The nameless is the origin of Heaven and Earth;
The named is the mother of all things.”
P.S. By the way, the entire Dao De Jing advocates thinking beyond symbols, transcending naming—not being constrained by it—and thus connect the reality and following the Tao, which embodies the will of love, frugality, and humility. Yet, few truly understand it fairly, as Laozi emphasized in his book. Perhaps we humans can transcend naming altogether eventually, with advancements like Musk’s brain-communication chip, we might soon discover how human wills are encoded and potentially move beyond the limitations of naming.
“Universal grammar” is a placeholder for the innate, domain-specific features of linguistic ability...
--— After looking up “Universal Grammar,” here are my thoughts: --—
Until we identify these features and prove that other species lack them, the concept of Universal Grammar remains uncertain. Only if we pinpoint what’s unique to humans and fail to help other species develop it can we confidently claim Universal Grammar is correct.
Personally, I think these innate features might not be entirely unique to humans. Even humans need sufficient social interaction and language exposure to truly behave like 'humans'. Feral children(https://en.wikipedia.org/wiki/Feral_child), for example, behave much like animals. I also imagine early humans had only a few words and symbols and acted similarly. However, if feral children or ancient humans were exposed to today’s social environment, they could likely learn our complex language quite quickly, because those words, symbols, and pronunciations are well-suited to humans for expressing our wills.
Perhaps, we humans are just lucky to have found effective ways to express ourselves —- symbols and words as we know them now -— and to pass them down through generations. This has allowed language and cognition to grow exponentially, giving us an edge over other species.
P.S. Our failure so far to teach animals structured language is exactly why I admire projects like Google’s Dolphin Gemma. To succeed, we need to tailor teaching methods to their species-specific traits—starting by understanding how they express their intentions/wills, and then hopefully helping enrich their naming systems and overall communication. We might even adapt this to help them learn from human contexts, much like large language models do.
Other animals have vocabulary. Anybody with a pet can easily understand that lots of animals can associate words with objects and actions, and Koko and Alex and all the other examples of talking animals demonstrate that animals can communicate their own wants with words.
The thing that makes humans nearly unique in the animal kingdom is structured language. I don't need separate words for "leopard in the bush", "snake in the bush", "leopard in the tree", and "snake in the tree", because we have grammar rules that allow me to combine the words for "leopard" and "tree" with a distinction of how the leopard relates to the tree.
I say nearly unique, because there is some evidence that whale songs are structured.
> The thing that makes humans nearly unique in the animal kingdom is structured language.
But what is “structured language”? I would argue it’s just another layer of naming—naming applied to already named names.
If animals have the ability to map leopard, snake, bush, tree in reality to names in their naming system, then why can’t they apply that ability again—create a new name like ‘leopard-bush’ or ‘snake-leopard’, simply by combining the existing solid names in their system? (Similarly, words like “in” could also be added.)
You might say they can’t—because they lack grammar rules. And I’d argue the issue runs even deeper: they don’t have a valid naming system to begin with, nor the ecological fitness or incentive needed to develop further naming.
If they can name ‘leopard’ within a valid system, that name should be reusable. They should be able to work on those names—as objects themselves. The process is the same. It requires the same abstraction: treat a thing as a nameable unit, then name it again. And that already starts to look like compositionality—or grammar.
The real difference is that humans ended up with a robust naming system. We got lucky—we gradually invented stable symbols that let us name on top of previous names. We could write them down, save them, pass them on—and then build new names on top of that naming recursively, iteratively, and divisively.
And that’s likely where the fitness gain came from. It’s a lucky, genuine self-bootstrap story.
Making compound words is not an important difference between human language and the languages of other animals. Even in human languages the ability to make compound words seems to have appeared later.
The main difference seems to be in having sentences structured into verbs and their associated nouns (which may encode various roles, e.g. patients, agents, instruments, beneficiaries, subjects of intransitive or copulative verbs, nominal predicates, results etc.).
As far as we could determine until now, the language of most animals consists only of a set of nouns, which typically have an associated implicit action.
For example:
"Lion!" (meaning "Climb up the tree!");
"Eagle!" (meaning "Take cover under branches!");
"Children!" (meaning "Come here, to mother!");
"Bananas!" (meaning "Come here to eat good food!");
and so on.
Some apes have been trained to compose very simple sentences, as complex as agent-verb-patient, but it is not known whether a similar form of communication exists between wild apes.
They tried that with Koko, it didn't work. It did not make the gorilla any smarter, it was an animal, and remained an animal.
If you read the criticism about the Koko project, it's that Patterson prompted her to make certain signs. I watched some clips of the signs, and it's very obvious that that is exactly what she did.
Animals can only communicate in a limited fashion. They do not have some "hidden" intelligence that you are trying to find.
> They tried that with Koko, it didn't work. It did not make the gorilla any smarter, it was an animal, and remained an animal.
I really doubt both their methods and your conclusion. The project tried to teach animals to adapt to the human naming system, and more importantly, it wasn’t a true social-level experiment with animals.
(That said, in my opinion, Koko was smart. Critics of Patterson’s claims have acknowledged that Koko learned a number of signs and used them to communicate her wants and needs.)
Imagine an alien taking an one- or two-year-old human baby and trying to teach him to communicate with alien using vocal signals that humans have difficulty producing or perceiving—and, most importantly, where naming is not a social-level communication that boosts ecological fitness. That’s basically what they did with Koko. It’s like training an LLM model on text corpora where the loss function (fitness measure) is inconsistent or even meaningless—would that produce a useful model?
We first need to understand how animals naturally name things, and then enrich that naming system in a way that fits their minds—not force them to learn human naming. Most importantly, this naming should improve their ecological fitness in a consistent way, so that they can ‘feel’ the fitness of certain naming and evolve their minds toward it.
That’s why Google DolphinGemma is so remarkable. If they succeed, they might mimic dolphin-like communication in a way other dolphins truly understand—gradually introducing naming and concepts that improve evolutionary fitness: finding food, recognizing others, being happy, and identifying suitable mates. If recursive naming develops, I believe it could even lead to real cognitive evolution.
This is just a rough idea, but I truly believe in it-—based on self-meta-cognition about how my own mind evolved and works, as well as other observations. Unfortunately, I don’t have the power or energy to explore it deeply, but I hope someone working on animal cognition research will take a closer look at using large language models like DolphinGemma and dive deeper.
There is billions of human-written texts, grounded in shared experience that makes our AI good at language. We don't have that for a whale.