AGI is not an engineering problem but a research problem. John Carmack is good at putting stuff together but how good he is at coming up with novel concepts for an open research problem remains to be seen. Even the rocketry example that is hailed here as a success mostly wasn't. That doesn't make me happy, it would have been far nicer if Armadillo had succeeded, more competition in that space is better. But for all the work done it was more of an advanced hobby project along the lines of those guys in the Nordics than something that moved the needle scientifically.
Carmack is someone who has proven to be an almost unequalled productivity machine when working on medium-difficulty problems... Now, for the first time, we'll see if his approach to problem solving can also work on a truly difficult problem. I agree it's very much an open question.
Is that really true though? It seems more like he's good at a medium difficulty problems in a narrow subdomain of software development, which is saying something a bit different. I might even say he's good at hard problems within that subdomain. How transferable those skills are is the most salient point. Assuming peak genius level intellect (which, I don't know, maybe?) It would still take something like 4 or 5 years to reach expert level knowledge in such a complex domain.
Agreed. Deep learning has revolutionized AI and anyone hoping to contribute to AGI is going to have to master DL first, and probably a lot more AI like a variety of probabilistic methods.
That's a challenging learning curve that's not much different from earning a PhD. And then, to stand out in AGI, you're going to have to integrate a dozen kinds of cutting edge components, none of which are anywhere ready for prime time.
At this moment in time, I think any attempt at implementing AGI is going to be half-baked at best. For now, a Siri / Alexa that can do more than answer single questions will be challenging enough.
I actually don't think mastering deep learning is very difficult. Theres a gazillion papers and ideas floating around, but the core concepts, that actually work, things like batch normalization, gradient descent, dropout, etc are all relatively simple. Most of the complexity comes from second rate scientists pushing their flawed research out into the public in some form of a status game
> [...] but the core concepts, that actually work, things like batch normalization, gradient descent, dropout, etc are all relatively simple.
They may be simple, but it's controversial why they work. For example dropout is not really used much in recent CNN architectures, and it's just - I don't know - ~5 years old? So people don't even agree what the core concepts are ...
Sure, this is true. I just threw dropout in there without thinking much into it. The point is even if we include the techniques that have been replaced by newer ones, the total number of techniques is small. Also if youre learning deep learning for the first time, understanding why dropout was used, and then how batch normalization came to replace it is key to understanding neural networks. Same can be seen in network architectures, tracing the evolution of CNNs from VGG16 -> ResNet and why Resnet is better exposes one to the vanishing gradient problem, shows how the thought evolution happened, and gives hints to what could be next/builds intuition for the design of deep neural nets
Get some basics of linear algebra down. Eigenvectors, Eigenvalues. Nail down Matrix Factorization, Principal Components, and the relationship between the two.
Learn softmax, logit function, different activation functions. When to use them. Difference between classification, binary classification, multi label prediction etc. Theyre all similar, just use a few different functions in the neural net
After this, go through some optimization theory and learn the different algorithms for optimizing neural nets, i.e. Adam vs RMSProp.
Then I would just get a list of all the top network architectures, then go through their white papers. Do this chronologically. Start at ~2012. Basically all the network architectures build on each other. So take the first good working deep CNN (alexnet), find out why it worked. Then move to VGG, why did that one work? What problems were solved? then move onwards.
^Do this for computer vision, then again for NLP (Word Vectors) and transformers (BERT, XLNet, etc).
Then youre done.
Theres also GANs etc, but that stuff is extra.
From there, choose whatever specialty you wanna research, and just grab the state of the art.
Nobody will want to work in a research program under this guy. He's just way too mean, and being mean is a no to researchers. Also, as other commenters have mentioned, this is a demotion.
Please take this as nothing more than my subjective opinion: I believe that humans don't have GI but HI - we have a "world-view" that is very idiosyncratic to being human, which is essentially heuristics all the way down - in other words, I don't believe there is a magical novel concept that explains HI, but that it is a collection of party tricks that evolved over time, i.e. hacky engineered system.
I don't see why that's not GI. That HI enables us to do incredible things, far beyond what we evolved to do (we evolved to hunt and fuck and exist in small groups, not to do quantum mechanics).
It begs the question, though: is the development of AGI contingent on some magical breakthrough, or is it a matter of endless tinkering and cobbling until we've got something that works?
It's probably both. There's going to need to be a series of breakthroughs, but there's also going to be a lot of engineering required. I believe that AGI won't happen suddenly. It will require putting a lot of pieces together. We'll get systems that have an incrementally better and better model of their environment.
Personally, I find it kind of offensive how scientifically-minded people believe they have the monopoly on generating ideas and making the world progress. That's clearly not true. Deep learning research wouldn't be where it is without GPUs, and compilers like TensorFlow and PyTorch. Engineers are huge drivers of change, they make things happen.
Deep learning research already involves a lot of trial and error. Tinkering and cobbling as you put it. People can't really tell, just writing things on a whiteboard, whether it's going to work or not. There might be some mathematical intuition, but a lot of it is throwing things at the wall and seeing what sticks, empirical testing. Some very high percentage of the research being done is basically thrown away.
Furthermore, I would personally say, as someone who works in deep learning, that we're collectively getting a little myopic. We finally got neural networks to do cool things. People are very excited, but they're forgetting neural nets are not the only kind of machine learning technique around. It actually works really poorly for some things. We're just largely disregarding every other approach because we have this one cool new toy. So, I don't know, maybe the next huge breakthrough will come from someone who's a deep learning outsider, and who's not completely locked into this paradigm and unwilling to look at anything else.
Yeah I believe in this too. It's not exactly politically correct to say so for a number of reasons, but if you listen carefully to some public intellectuals, behind the scenes they believe the same. For instance, noam chomsky will talk about how the ability for humans to speak language is hard coded. That if we really were some piece of clay to be molded, noone would be able to run their own life.
IMO he's wrong about at least the importance of it, you can point to many tasks that are clearly not hard coded because they are recent inventions yet are difficult for a general AI (with zero task specific architecture) to perform, e.g. driving, playing jazz piano, playing any video game, doing advanced mathematics. Either language is much harder than all of these things (I think that's wrong) or all of these things require language capacity (in which case it's just another form of 'language = thought').
I believe this too, I think everyone in the field should probably first study the biology, workings and evolutionary mechanisms behind it. I think you first have to understand why actually such level of "GI" exists in humans, what's the actual purpose upon which it evolved. In a perfect world without constraints we would never exist (or life at all), because there would be no constraints that shaped us. Our mind and the way we think is built around the world we live in. Without all the sensor (and other chemical / inside the body) inputs the brain would not work - it would stop to work after the body and inputs are detached (even if you kept supplying all the nutrients). I don't think you can just "store" or encapsulate GI on digital storage because you would have to emulate all the complex environment inputs which it needs to function in a comparable way to humans... We are the "product" of the environment we live in.
Bringing up armadillo wasn't meant as an example of success in business but rather as an example of ability to dive into new fields effectively. They did some cool stuff in their short run, and there's an offshoot company still going.
But yea, I agree with your general point. I'd just note that having that ability to be insanely productive in working on things people haven't done before means to me that if it's possible for someone like him to really get good at this field, he's probably gunna do it.
Who knows how far you can get with just "putting stuff together." That's what Edison did.
I'm not in Carmack's league by any stretch of the imagination but I earn a living with being able to absorb a lot of data on a new field in a very short time so I have some idea of what that is like. When other people have already done all the hard work for you that's very easy going if you have some basic knowledge that you can integrate the new stuff with. But that's an entirely different matter compared to actually moving the needle on new stuff, this takes years at a minimum; and is not something that you can do by reading up and then rolling up your shirtsleeves. If only it were that easy. There are 100's of John Carmacks in various fields, I've met a couple and while in the past this sort of attitude was a prerequisite to being a scientist (in the 1800's every scientist was pretty much a polymath, there wasn't all that much knowledge to begin with) nowadays any science worth doing is going to require a lot of specialization first.
This is akin to the way - on topic - computer games have developed; in the early days almost all games were made by individuals. Now it is all studios and teamwork, very rarely does an individual still manage to break out of the mold and the level of expectation that we've set. But when it does happen (Minecraft, for instance) it can be a runaway success.
Anyway, I wish John the very best but I think the chances of an Armadillo repeat are somewhat higher than of him coming out of his closet with a working AGI, and just in case he does I'm not sure the rest of the world is going to be ready for that (entirely different discussion, there are plenty of SF books and movies exploring that theme).
> nowadays any science worth doing is going to require a lot of specialization first
This seems like one of those seemingly-obvious points that everyone believes but has a decent chance of being proved completely wrong by an unexpected discovery or breakthrough. Assuming I understand you correctly; knowing which giants to stand on has consistently proven to be a requirement - both in science and engineering.
But I see you're sort of alluding to this at the end of your comment, where you're not dismissing this effort as meaningless, but rather not a guaranteed success.
Such pointless gatekeeping, if Carmack turns his attention and resources to AI he will be able to make contributions to the field. Will they be groundbreaking? Maybe not, but why is everyone so eager to immediately discourage him.
> When I think back over everything I have done across games, aerospace, and VR, I have always felt that I had at least a vague “line of sight” to the solutions, even if they were unconventional or unproven. I have sometimes wondered how I would fare with a problem where the solution really isn’t in sight. I decided that I should give it a try before I get too old.
A lot of what he did in PC graphics was very original, but in the old days it was narrowly applicable, while by the time Doom 3 was in development, his gfx programming could be more widely applied to other things. I think his fast inverse sqrt was VERY impressive, which he didn't invent first but may have come up with independently.
Even the rocketry example that is hailed here as a success mostly wasn't.
This demonstrates how hard the problem is. When you're tackling a really hard problem "mostly not a success" is a success. Most people faced with the same problem would return "no successes".
AGI is not a research problem but an imagination problem. I can’t vouch for how good John Carmack is in imagination, but striving to put things together with a goal seems like a good place to start.
I can imagine AGI just fine. Doesn't get me one tiny little bit closer to being able to make one. There are several ways in which one could go about such a development, all we have for now is an existence proof and none of the paths pointed out so far have been viable. Whether John will come up with novel path is not really an imagination issue but one of very deep understanding of the problem space, what has been tried so far, why it did not work and then to come up with something that either fell through the cracks as non-viable and then to recycle it in a way that it is viable (the current neural net applications are like that) or an entirely novel approach. The latter will likely come from an outsider but it would be an extremely lucky shot to hit something workable; the former may be a possibility worth investigating.
The reason why the latter has some chance is there are sometimes approaches tried early on in a field that can't succeed because something else needs to be invented first; or the computing power required is prohibitively expensive. Carmack's skills and ability to absorb knowledge might help to spot such an opportunity.
The parent's point was that reaching AGI isn't merely a research problem, in the sense of X amount of person-hours of research by typical researchers will solve the problem. Rather, we need new ideas and new conceptual frameworks, i.e. new imaginative leaps to reveal the path forward towards AGI.
Could you list a few fields of research where "X amount of person-hours of research by typical researchers will solve the problem"? I can't think of one.
It's not about fields specifically, but about particular problems within fields. An example is neuroscience discovering the function of some unknown functional unit of the brain. We have all the conceptual machinery to solve the problem, we just need to fill in the details. On the other hand, the problem of consciousness doesn't even have the conceptual machinery in place such that more details will lead to the solution. A solution here will require conceptual leaps that we can't put a boundary on like we reasonably can when the conceptual groundwork is already established.
In mathematics the word used for these kinds of problems is "inaccessible", e.g. Reimann Hypothesis or (previously) Fermat'a Last Theorem. I don't know if Carmack's chances are as good as Wiles' were but certainly better than the average joe. It's also the case that AI is a substantially younger field (arguably it was only possible to correctly evaluate ideas since powerful GPUs were released this decade) and so the difficulty of open problems including AGI is not yet known.
It would be great to see what Carmack could contribute to Cyc, not just his own common sense (which would be great too), but deep rich useful interesting applications of commonsense knowledge (including but not just games).
