> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates?
> Why are some people willing to free solo climb icy mountains despite the danger?
> Why are some people willing to have fewer children in order to pursue career goals?
> Why are most billionaire tech CEOs interested in space travel and extraterrestrials?
> There are no doubt potential evolutionary explanations for each of these phenomena.
No doubt?
> But, from a purely evolutionary standpoint, these phenomena are at least (a) not obviously explained or (b) seemingly irrational.
Correct.
Many phenomena have an evolutionary explanation. However some phenomena do not - random mutations happen and things pop into existence without much connection to mating, survival etc. Trying to shoehorn evolutionary narratives onto them is not letting the evidence fit the theory, but instead trying to get facts to fit some preexisting theory.
Some traits just pop up due to random mutations, and trying to find an evolutionary mating/survival/etc. narrative for each of them does not make sense.
Even beyond random mutation, it also makes sense to think about behavior in totality. Curiosity and a willingness to do insane things seem to be tied up in an overall system that does very well in maximizing reproductive success. But this doesn't mean that literally every behavior that results in individually optimal at spreading genes.
Human backs are maybe a good simple example of this: walking upright has been great overall, but has a lot of drawbacks that we suffer from such as difficult childbirth, back problems, etc. You could ask "what is the evolutionary advantage of lower back pain" and the answer of course is there's none, but there is still a broader explanation that makes sense when you consider the whole system.
It's funny because I have been a "gym rat" much of my life and until very recently wasn't motivated by vanity, specifically, being attractive to the opposite sex. I've seen a burst of assertions lately from 20-somethings that seemed to deny anyone would have any other motivation for physical fitness.
Oddly compared to my cohort I am both a nerd and a jock. I find intense exercise be it hiking or cardio or weights helps me keep my sanity in terms of sitting all day and the other stresses of my work. It's not like it is abstractly good for my health in the long term and I'm discovering now that it probably wasn't but that I feel better immediately in the short term from physical exercise.
I perform hard physical fitness for a lot of reasons, but the primary reason is simply that I enjoy it. If I skip a workout I miss it and don't feel well.
Atomic Habits touches on this. People think that I have all this discipline to get up and workout, but the reality is that I enjoy getting up early and working out. I enjoy training JiuJitsu and more often than not getting my ass kicked. I can't really explain why, somewhere along the way I associated feeling good with hard physical work.
I am 100% not motivated by vanity. When I was 25 years younger, yes, I was. But now its mostly to stay in shape to live longer and more comfortably. It's just that. At this point in my life, increasing my attractiveness to the opposite sex probably just adds problems to my life.
Ditto. I think it's like doing gardening, or jugging, or riding a unicycle (waves at shannon). The change of mindset and switch in cognitive usage whilst still taking advantage of "success" attributes like discipline, organization, ability to project manage onself/setting milestones, allows me to actually focus better on my job, when I get back to it from my "hobbies".
And I'm also sure many will also have encountered that moment when you have really focused and dug deep into a problem and hit an impasse, then force oneself to take a break and do something completely different, and in the midst of it all, "aha!"
I wore a heart monitor for a month and we observed about 30 seconds worth of AFib right at the very end. (Oddly I had experienced a very interesting setback in my personal life around then.)
I had the signs described above. My doc said I should go on exercising but I shouldn't train more than 1 hour a day. (In a time when my life went to crap I had got into the 2 hours a day habit to fight my depression.)
That sounds like it might just be correlation. But besides that, athletic training can cause left ventricular hypertrophy but it hasn't really been associated with disease state afaik
Reading through the whole linked article seems to suggest precisely this too. Unless I'm missing something, all it's saying is that if you exercise routinely for >1 hr at a time it's more likely that a doctor may mistakenly diagnose you with a cardiovascular disorder because of the characteristic changes in your heart as a result of repeated exercise, especially if they're not aware that you do exercise often. So they have to be careful when evaluating patients who exercise for heart conditions so as to not misdiagnose a collection of otherwise benign characteristics as a more severe disease requiring treatment. In short, athlete's hearts are a little different; evaluate patient with this in mind.
People who run marathons competitively have atrial fibrillation at a several times higher rate than other people.
It's an interesting problem for competitive athletes (say Serena Williams) who might not play as hard since the risk of concussions are worse if you are taking anticoagulants.
Ooof. There's my yearly reminder to go check if Alex Honnold is still alive. My heart skips a beat as Bing Search autocompletes "Alex Honn" to "Alex Honnold Death". But he's still alive! And he's married now.
"Individual organisms are best thought of as adaptation-executers rather than as fitness-maximizers" -- John Tooby and Leda Cosmides
I would say the evolutionary explanation for each of these is actually pretty straightforward. We live in an environment that's very different than the one we were in for most of our evolution. Our evolution hasn't had nearly enough time to account for the modern environment.
> Why are some people willing to have fewer children in order to pursue career goals?
The obvious explanation here is that there wasn't really birth control when humans lived as hunter-gatherers. You couldn't make that kind of sacrifice for the most part. There also weren't really careers in the modern day sense. Evolution hasn't had a chance to account for birth control yet. It's only been around for 60 years or so.
> Why are most billionaire tech CEOs interested in space travel and extraterrestrials?
Because all life has some instinctive drive to explore and spread. By exploring, you have an opportunity to find new resources and territory, which is essential for survival or to outcompete other life.
> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates?
When humans lived as hunter-gatherers, there were no gyms and no muscle building magazines, no cult of bodybuilding. For the most part, you got as fit as you needed to be to hunt animals, to fight, and build whatever you needed to build.
However, I think the part about being "well past the point of being a way to attract potential mates?" is a value judgment. In actuality, growing a lot of muscle mass is a way of showing off your determination and fitness and standing out. Male bobybuilders probably have a much easier time attracting mates than the average joe.
> Why are some people willing to free solo climb icy mountains despite the danger?
Evolutionary drive to explore. Evolutionary drive to show off prowess to attract mates and gain status. It's a high risk, high reward strategy.
I think these "overdriven goals" might be the result of leaky abstraction. We are basically seeing an implementation detail come to light. As TFA points out, there is really just one reward-taxis algorithm. Everything drive-related is a variation on it. Meaning there is no "obtain visual appeal via body building until sufficiently attractive" algorithm.
There is a top-level algorithm of "find mate and reproduce", which is enjoyable. How does it boost the odds? By doing things to be more attractive, for example. But how does one "know" one is on the right path? Activities which we learn increase attractiveness are enjoyable. There's a gradient we follow. More attractive = more enjoyable. So you end up with complex behaviors that basically become self-hoisting; the activity itself becomes the reward.
Basically, games and hobbies. Something becomes enjoyable vis a vis various learned reward functions, so we just pursue the enjoyable thing.
There's also a meta-loop wherein staying mentally stimulated and curious is itself a useful fitness, so we do things which are interesting, because it's enjoyable to do that.
I agree with your idea of leaky abstractions and overdriven goals.
I think one thing to remember here is that while for sure attracting mates must play a role in the appeal of bodybuilding, there's also a competitive game aspect. You can compete against yourself and your peers. This can appeal to instinctive drives to compete and gain status, maybe again past the point where it's reasonable in some cases (are you really winning at life, gaining something meaningful by doing this?).
If we just talk about the attracting mates aspect of bodybuilding, we also have to remember that if a man is trying to attract women by gaining muscle mass, his idea of what women in general might find attractive is just an approximation. Maybe this man found that he was gaining more attention from women when he started to gain muscle mass, so he just keeps doing more of that. Eventually he might gain even more attention for other reasons (improved self-confidence), but it gets hard to tease apart the real causal factors.
> Evolution hasn't had a chance to account for birth control yet. It's only been around for 60 years or so.
I'm no anthropologist, but I'm pretty sure various forms of birth control have existed for at least several hundred, if not several thousands of years. Granted, the invention of the birth control pill was a pretty big milestone, but it's not like nobody had ever thought of ways to fuck without getting pregnant before then.
If anything though the speed we are able to adapt is what is interesting.
In my own life I think of the sitcom "Head of the Class" as a kid. Basically, painted "nerds" and computer geeks as highly sexually undesirable. All signaling was that this is what not to do to get laid.
Didn't even take 25 years for that to essentially be reversed to the point of a computer geek being a type of peacock feather to flaunt and strut around.
IMO the point we are maladjusted for the environment we live in is highly overblown.
What? I work at a FANG and I can guarantee you there are very few peacocks there. I've had success on the dating market specifically through how I'm differentiated from my coworkers(into sports/outdoors, not obsessed with scifi/marvel movies, goes to the gym)
It’s tautological at this point to say that any behavior arose from an evolutionary standpoint.
It’s conveniently tautological because it’s impossible to prove or disprove without time travel :)
A more interesting conversation which is also impossible to prove/disprove is whether there is any behavior that isn’t just behavior/instinct. It sure feels like it with consciousness, but if humans are weights and biases, it was inevitable that I typed this :)
> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates?
Margin of safety. Many would probably disagree with the author's assessment that they're well past that point.
> Why are some people willing to free solo climb icy mountains despite the danger?
Conquering new land is deeply embedded into our psyches for evolutionary reasons. The species that's not afraid to do that will probably be more successful in the long run. Conquest of more land is generally correlated with better survival chances.
> Why are some people willing to have fewer children in order to pursue career goals?
Perhaps many, like myself, are aware of how expensive kids are but unaware of how difficult having kids gets later in life. So they optimize this wrong. I wish I had started earlier, but I put it off until later because I thought I'd be able to provide for them better. This thinking was correct, but having them earlier would still have been better, for me at least. The point is, it's certainly not due to a lack of evolutionary interest in having kids.
> Why are most billionaire tech CEOs interested in space travel and extraterrestrials?
> Conquering new land is deeply embedded into our psyches for evolutionary reasons. The species that's not afraid to do that will probably be more successful in the long run. Conquest of more land is generally correlated with better survival chances.
Overwhelming majority of people stay where they are. Generally humans need huge practical motivation to conquer lands.
Plus, interpreting climbing as conquering lands is super odd. It has more to do with any other sport achievemt then anything else.
This seems like an extraordinary claim. I can get behind the idea that it’s possible for a random mutation to impact personality traits. It is entirely different to turn to this as a general purpose mechanism for things that are hard to explain. It seems overwhelmingly likely that most traits are in some way selected for, because useful traits are naturally more likely to propagate than random useless (which will pass on, or not, at random)
I think our species is post-genome for a lot of the complex behavior you see in society. Cultural and ideological memes use us as information carriers. We fixate on certain ideals and values, and some of us make them our entire lives.
Preferring Pokemon over Halo doesn't entirely reduce to one's genes. Being too busy to have kids is situational and part of a complicated modern life. There is genetic contribution to these outcomes, but we're well past the point of genes being the most complicated factor. Now you have to model society and a person's random walk through it.
I'd like to add (a bit off topic) that humans are not necessarily capable of causal reasoning, at least not alone. It's a cultural process where the true causes are separated from correlations in time, with lots of experimentation. Only with special training and in specific domains can we reason causally, and most population just learns the surface never getting to real understanding. In that regard we're like language models.
So part of what makes us so smart is not in the brain, it's culture itself. If you observe feral people - they never get far on their own.
Great insight here. I wonder how changing culture could effect reasoning ability in the future. Perhaps there are critical aspects that might be important to protect or eliminate in order to avoid a return to feral nature.
Could this be one of the great filters that we, as a species, still need to pass?
I agree with most of what you said except for this notion that we’re “post genome”. We are absolutely not.
What you described is just “nature vs nurture”. Genes don’t make anyone prefer Pokémon over Halo directly, but the mechanism was built by evolution. A person who makes friends playing Pokémon is certainly likely to develop a fondness for Pokémon, because having friends is beneficial to survival and reproduction.
Modern computing is “post hardware”. That doesn't mean the hardware doesn't matter, or that changes to the hardware don't make a difference; it means describing the hardware is insufficient to describe computers, whereas describing the software is mostly sufficient.
This is a meaningless distinction, though. Of course you can’t describe a human’s behavior purely in terms of genome. Essentially no one holds the position that this is possible. You can’t even describe a bacteria’s behavior purely in genomic terms. Behavior is inextricably linked to the environment. The environment for humans is incredibly complex so the interactions between environment and genome are even more complex. You cannot describe human behavior in terms of just environment either.
Maybe this is an interesting position for software, because modern CPUs are explicitly intended to be general purpose and as such are all theoretically equivalent. Even there the abstraction falls apart pretty regularly, though, as Intel gets destroyed by M1 battery life and attacks like rowhammer affect some hardware far more than others. It’s only post Harare as long as the hardware it’s good enough for the purpose.
> Of course you can’t describe a human’s behavior purely in terms of genome.
Phenotype, whatever.
> It’s only post Harare as long as the hardware it’s good enough for the purpose.
Most human brains and bodies are good enough for most human purposes. Sure, there's large variation between humans, but the variation between populations isn't great enough to be significant, as far as I'm aware.
> Most human brains and bodies are good enough for most human purposes.
The difference is that we know how general purpose computers work. We literally know exactly how they work due to the fact that we built them, but also we have developed very useful abstractions for how to use them. We do not have an equivalent understanding of human behavior.
Humans are not general purpose computing devices that we know how to program. Humans are extremely complex systems that are already running software and we don’t understand that software. We’re engaged in black box debugging and we are nowhere close to actually understanding the software running on the box (and we’re little closer to understanding the hardware).
We don't understand fluid dynamics either; doesn't stop us from modelling it.
Also, I wasn't trying to equate computers and brains, or ideas and software. I just picked the example to show how “post-genome” can make sense as an idea. It's not an applicable analogy.
That’s right. We do not fully understand fluid dynamics. We are still trying to model it. And we are certainly not “post fluid dynamics” even though we successfully send water through pipes and put planes into the air.
We are “post-molecules” for most fluid dynamics behaviour. Though I don't see how fluid dynamics is relevant, here; I'm using these as examples where your general argument falls down, not analogies.
I didn’t make a general argument. I made a statement about being “post-genome”.
Let’s go back to your original comment:
> Modern computing is “post hardware”. That doesn't mean the hardware doesn't matter, or that changes to the hardware don't make a difference; it means describing the hardware is insufficient to describe computers, whereas describing the software is mostly sufficient.
You are correct about software. Describing it is largely sufficient to explain computer behavior.
This is not the case for human behavior. The genome encodes the hardware, but also encodes a ton of critical software (call it firmware?). Even basic needs like eating are not explained by culture/society. More complex behaviors called out in the article are also not explainable solely by cultural influence. Certainly culture influences the gymbro to build muscle, but it cannot be solely that because most people do not build massive amounts of muscle. If culture were solely responsible you’d expect to see this influence people far more evenly. Same for climbing mountains. There’s very little societal push for this. And I mentioned weddings already. You can certainly explain the specifics of weddings culturally. But you cannot explain why people desire to pair bond culturally.
We’re not post genome because we cannot explain most human behavior by looking solely at cultural influences.
But are those traits behavioral, or are they the mechanism that give rise to flexible behavior? Let's take language. Imagine saying that linguistic components are naturally selected for in the genome. It's the wrong level. Language happens at the level of brains and societies, not genes. Genes are used to build proteins which are used to build brains, but brains interact and learn from their environments, and even shape their environments, causing feedback loops.
> But are those traits behavioral, or are they the mechanism that give rise to flexible behavior?
I don’t know what you mean here. The specific traits mentioned (building muscle past the point of attractiveness, climbing deadly mountains)? Traits in general? Something else?
> Imagine saying that linguistic components are naturally selected for in the genome. It's the wrong level. Language happens at the level of brains and societies, not genes.
It’s not the wrong level. From what we can tell, other animals are literally incapable of complex language. That’s genetic. Language traits literally have been selected for in the human genome. We have a brain that can process language and a tongue structure capable of making complex sounds that other animals can’t apparently replicate. Certainly language also has environmental/societal components. But it’s inaccurate to say that it’s not also genetic.
Unless you mean that literal languages are not genetic, in which case, sure. There’s nothing in the human genome that encodes English. There’s also nothing in the human genome that encodes the taste for yogurt, or the drive to be the fastest runner, or the Catholic wedding ceremony. And yet genetic components still certainly underly those in a meaningful way. It’s worthwhile understanding the genetic components even if they cannot explain the entire phenomenon.
This is also ignoring a crucial aspect of human behavior: we have moved past simple evolutionary optimization. Our behaviors can now change very rapidly due to spoken and written language. In The Selfish Gene, Richard Dawkins called this "memetics". E.O. Wilson also wrote about this, I think referring to it as "culture" (I don't remember in which book, sorry). Evolution is still necessary for understanding human behavior, but it is not sufficient by itself.
This seems like something Buddhism attempts to address (from my understanding)
> Desire is like an never-ending thirst
> No amount of success ever will quench it
> Their solution - Remove desire.
The only way to get everything you'll ever want, is to want nothing.
Novel behavior and random mutations do have an explanation - they help explore possibilities and find better ways. Most of the time it's a loss, but it's still worth it in the long run. It's also a desirable sexual trait to have unusual abilities or achievements, to be the explorer or the conqueror (for men, for women there are other criteria).
> Many phenomena have an evolutionary explanation. However some phenomena do not
Some traits might just present themselves as {seemingly nonsensical activity} in an industrialised world. Where back in time, these same people might have an advantage in {seemingly sensible activity}.
This does not preclude random traits being passed on, it is merely another way too look at things.
> and trying to find an evolutionary mating/survival/etc. narrative for each of them does not make sense
It is also a very lazy and uninteresting line of thought.
Of course {trait} had en evolutionary advantage. Why else would it be here? And if it is indeed random, then it may be of slightly more interest, but tells me very little.
We’re observing shadows cast against a wall. Eagerly pointing out this and that pattern. When the interesting part is knowing the individual shapes that make up the dancing display.
Thanks for quoting this part of the paper. It's probably the most interesting part.
To add to what you said, though, I don't think of these phenomena in terms of random mutations. In other words, I don't think of these people as anomalies that do anomalous things; moreso, I think of it as following our growth imperative (i.e. the expected reward gradient), which, if we follow it long and hard enough, can lead to behaviors that appear anomalous.
One commenter below (kortex) really hit on what is at least my interpretation when he said "we are basically seeing an implementation detail come to light."
Trying to find an evolutionary explanation in existing behaviors is putting the cart before the horse. Organisms can do many things that have nothing to do with evolution. However, evolution will take care that only some of them will survive. So, what was good for evolution can only be inferred from the past.
Also, probably obvious, but us humans have been countering evolution for a while now. Medical science (both for sickness and fertility) + modern lifestyle keeps people alive that wouldn't otherwise be if we were still living in the woods.
So (at least from my armchair) it's no wonder we see more "pointless" behaviors.
This isn`t at all countering evolution. It is changing the fitness pressure. being able to parent effectively with smart phones and sedentary jobs and needing a lot of education is more fit than not having metabolic disorders or a weak immune system.
If anything our tendency to not have reproductively isolated subgroups and large population is making speciation less likely. But rest assured our genome is slowly changing as we change our fitness landscape.
You completely ignored the "4e. What Space Are We Navigating?" and are thinking in terms of navigating evolutionary fitness/random mutation. I can see how our large brains create a new kind of virtual space we move around in, and it is unlike either of the above.
We're navigating "survival space", the space of all possibilities of existence in the future, under limited resources. It's pretty open-ended, we don't know what we could be, the peaks of evolution are clouded and we don't know if any particular path leads to it or is a detour. Only when you're close enough can you make progress by going directly towards the target.
I see survival space like an outer hard boundary. Brain can and in reality often makes its own vaguely related space and we navigate it oblivious to the hard survival boundaries..till they hit us.
Call it "linear regression brain" -- the inescapable need to consider minute, limited traits as having some fixed, measurable contribution to the outcome, rather than a component in a rich tapestry of complex dynamics.
Of note, evolution is not equal to natural selection. "random mutations" are causes of evolution, drift (random) or selection (fitness benefit) are both mechanisms by which mutation can manifest in a population.
any trait can be positive, negative, or neutral for evolutionary fitness. If a trait is neutral it can potentially survive and be passed down.
There is evolutionary fitness at an individual level, but cultures have evolutionary fitness as well. A culture might preserve individuals that have low individual fitness because they create cultural fitness.
For example the need to explore and expand the boundaries of knowledge may not be useful at an individual level, however a culture with those individuals might survive better than a culture without. All members of the culture that does not have those individuals may die, even if individuals are very fit.
Sociopathy is likely conserved at a cultural level because cultures periodically need sociopaths to completely obliterate competition or ruthlessly compete to survive.
Cultures also are constantly spawning mutations (civil rights, black lives matter, nazi ism, capitalism, democracy. social justice etc) those mutations can be positive, negative or neutral. Neutral traits can hang around if they carried alongside traits that do provide cultural fitness.
is there not an additional level added with mental abstractions?
e.g. i blast it at the gym (i dont actually but play along ;)) and i build an abstraction of life that that is part of. the fact that the abstraction has limitations is irrelevant. we dont process the real world we process our learned abstracctions of it.
> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates?
Being a gym rat is not a heritable trait, but the ape who keeps doing whatever they were doing that made them strong and sexually desirable could be much more evolutionarily successful than the ape who stopped and became weak and undesirable. More generally, a positive feedback loop for beneficial behaviors is a desirable trait.
> Why are some people willing to free solo climb icy mountains despite the danger?
Being a free solo climber is not a heritable trait, but exploring your environment and incrementally pushing one's limits are both desirable traits. The ape that doesn't know which rocks he can climb falls to his death, the one who knows where to place their hands survives to reproduce.
> Why are some people willing to have fewer children in order to pursue career goals?
Corporate ladder climbing is not a heritable trait, but investing more resources in fewer offspring is advantageous when the odds of those offspring surviving are high. Even with no kids of your own, you pass on your genes through nieces, nephews, etc. The desire for resources has obvious evolutionary benefit.
> Why are most billionaire tech CEOs interested in space travel and extraterrestrials?
Neither being a tech billionaire nor being interested in space is a heritable trait, but the desire to explore is undoubtedly a desireable trait. The apes that expand into virgin territory can acquire more resources and become more resistant to local catastrophe than the apes who remain in one location. Elon Musk just has much better odds of becoming king of mars than I do. 500 years ago this may have manifested as building big ships instead of big rockets.
Of course random mutations happen, but those random mutations get eliminated quickly if they are harmful and spread quickly if they are helpful. Saying something is just due to random mutation is not only an unsatisfying answer, it is the answer to the wrong question: it doesn't matter why we started doing it, why do we still do it? Evolution doesn't create any trait, it merely perpetuates or fails to perpetuate them.
Not all behaviors are hardcoded into our DNA, certainly not at the level of specific actions we typically take day to day - my ancestors did not pass on their genes more successfully because of their affinity for arguing on the internet - but those traits which are heritable are unavoidably influenced by evolution. Evolution is the null hypothesis - if a system left to its own devices would do something without any external intervention, why invoke any additional explanation? It is those saying that something is genetic but which has somehow defied evolutionary pressure which are asserting a narrative, and who must show evidence in support of that narrative. Some narratives are legitimate, for example the traits of domesticated plants and animals aren't explained by natural selection; but oftentimes it is just "magnets, you can't explain that!"
Anything can be "just" anything else if you simplify it enough.
The idea that humans are goalseeking systems motivated through a single parameter of dopamine response seems like it might be reasonable enough, but that then just moves the problem around: how is a particular set of stimuli + past stimuli linked to a dopamine response?
Furthermore, are they intending to force all behavior analysis through a control system with a single parameter? Can we ask questions like "what is the frequency response of the dopamine system?"
It's definitely not a single parameter. I've been down this rabbit hole a bunch.
There's actually 5 main subtypes of the dopamine receptor. They express in different concentrations in different brain regions, and have different functions.
DRD2 is kind of the heavy lifter when it comes to goal seeking. If you had to boil it down to one idea, I'd call it "go/no go". It might be tempting to try to map D2 to something like a coefficient, perhaps even the "run coefficient" in run and tumble. But it's not at all like that, even on its own. It's very nonlinear. Furthermore, the action of D1 modulates D2 in a concentration-dependent way. So you have not just D2 signaling, D1 signalling, but also the differential signal as well. We don't know exactly how it maps out, but we know it plays a roll in weighing risk/reward ratios. Screwing with this pathway often results in "low risk low reward" mild manner folks switching to "high risk high reward", often going into gambling, manic spending, hypersexuality, and grandiose thinking. This can be seen occasionally with e.g. Parkinson's drugs such as L-DOPA.
Actually my headcannon is there are some parallels between Parkinson's and control theory. Net loss of dopaminergic neurons results in lower bandwidth on your control loop. In PID loops, one of the symptoms of insufficient I/D gain is ringing, where the controller constantly hunts for the setpoint and overshoots. Tremor is a common symptom of Parkinson's (though it affects much more than just tremor), but if you notice, it's usually not a continuous tremor, it's only when trying to execute motor control, especially fine motor control. The body knows kinematically where it wants to be in space, but the feedback loop rings constantly as it isn't able to get sufficient bandwidth to error correct.
Anyway, I think the point of TFA is not to suggest that this loop is a single parameter, it's just to model it as such. It's a spherical cow reduction. That kind of reduction lets you build practicable theories and test hypotheses. Such as: if Parkinson's is the result of reduced bandwith, can we treat the symptoms by increasing dopamine gain? And in fact, we can, that checks out.
The danger of people falling in love with an analogy. In simpler systems (cellular), it may be straightforward. In mice, you can say that their movement follows algorithmic predictions when only one goal is present (food), but I am sure that if you enter a second goal (avoid predator), and a third (procreate) into the same experiment, you will find that the algorithm will be insufficient...at least insufficient until you have some data reduction technique on the problem space.
Another question is - what stimuli, changes in environment, or thought patterns affect dopamine levels and reuptake, and how? Also how do other neurotransmitters attenuate or floor/ceiling our behavior gamut in a way which modifies how we react to or are informed by dopamine levels and their rate of change.
Essentially, if dopamine can be boiled down to a single lever, then I think that lever has as many buttons as a Formula 1 steering wheel.
The major problem with this argument that it relies on a very important (yet questionable) assumption: the reward hypothesis, the idea that the maximization of the expected sum of a single signal variable describes the organism’s behavior (http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhy...). First we need to be careful defining what an “organism” actually is (because we are actually made out of billions of systems interacting with each other at the same time), but it is also hard to believe that all those interlocked systems move towards a single purpose of maximizing one variable, whether it be by chance or by coordination. The other problem is that the researchers might all be performing the error of defining the reward function based on our own narrow views and try to shoehorn it on other species as a process of pattern-matching. The third is that, as shown in the bacteria example, the organism and the environment are not independent from each other, and what we call “intelligent behavior” is not an inherent aspect of a creature’s “algorithm” and cannot be studied independently in isolation from the environment. (These can all be extended as a critique of evolutionism as a useful scientific theory; the problem is not that it’s wrong, but that it becomes more of a tautological statement as you bury yourself deeper into it, so its usefulness as a theory needs to be questioned.)
A bit more tangential, but the article really falls apart when the author starts talking about examples related to humans, using the careerism of current middle class society as an example (The notion of a “career” is actually pretty recent from an anthropological perspective! Those anecdotes about hierarchical milestones and pushing ourselves for career advancement has more to do with the culture and ideology of today’s world than any other weird evolutionary/neurological reason!)
Pretty sure even different subsystems of my CNS will push in different directions on that one. One just waits for the networks to settle and see what your choice is this time. Possibly biasing the outcomes with thoughts like, "what a week" or "I was looking forward to meditating"
not going to get into the latter part of your comment, but in response to the former part, I haven't found a way to disprove the hypothesis that organisms' behaviors are all described by the attempt at maximizing the single variable of "number of possible future states" ala https://www.ted.com/talks/alex_wissner_gross_a_new_equation_...
… and I’m not convinced. It’s basically claiming that maximization of entropy over all possible paths in a time horizon leads to emergent cooperative behavior in some kinds of simple toy puzzles, but then claiming it as the center of human cognitive behavior is a far stretch.
It's far from being _proved_, but the more I've thought about it the more it seems to intuitively line up with other systems of behavior optimization
But yes, its actual application to, say, social psychology has not been rigorously (or even partially) established.
I have a fantasy that a grad student or two will go down this rabbit hole to see if it can be used to build models that hold water. But so far I haven't seen anything.
I kept expecting the phrase "reinforcement learning" to show up, but even after mentioning a Bellman-style time-decayed sum over future rewards, it doesn't appear, which seems odd to me. And roughly, I think "runs" == "exploitation", "tumbles" == "exploration" in terms other people use.
A thing that the author glosses past is the relationship between having a model of the world and maximizing rewards. It's taken as given that a model-based approach leads to better predictions about rewards ... but when we try to get systems to actually solve problems, this often isn't the case. Learning policies which yield high rewards is often easier than learning a model which actually describes the environment. I think of value iteration as learning "I feel this is a good state", policy iteration as "I feel this is a good action", and a real model as implicitly providing "I think this is a good action because I expect that ..."
If artificial systems have the greatest successes with model-free approaches following algorithms very similar to the one the author describes, but humans seem to have a rich model _and_ do better than those artificial systems (at least accounting for data efficiency), doesn't that at least suggest that we're thinking differently?
I think it really just suggests the models are being trained improperly. We just don't train alogorithms on data as complex as the real world. So, it's not surprising that simpler (less correct) models win out over more accurate nuanced ones.
My take is more that the framing of maximizing an externally provided reward describing a single task only helps an agent who can rely on the rules of that task remaining fixed. Humans don't just try to do one task well; we switch between tasks. We don't have episodes that reset really; we have one long episode, and our actions decades ago can have bearing on our well-being today. What is a good state or a good policy varies substantially over time. The concept of the value of a state or (state, action) is ambiguous, because who knows what you'll want tomorrow? If the definition of value shifts, but the rules of the world don't, then a modeling becomes more "valuable".
Also, now that I think of it, although we often talk about how much more data-efficient humans are compared to ML models, we could say that we're actually much more data-inefficient than most (or all) other animals. Our gestation period is long and then we're totally useless as a baby. We require a lot of learning and development early on in order to reap the benefits later (which consist of generality and flexibility like you mentioned and also, for example, being able to generate our own synthetic data via imagination and counterfactual reasoning).
Model-free systems are easier to build, but researchers are working on model-based approaches to reinforcement learning and I remember seeing blog posts last year describing some promising results.
It sounded like it was still reinforcement learning, though.
I don't think it makes sense to tie dopamine to physical movement and analogize that to "movement" in the ludicrously high dimensional and topologically weird "space" of all possible actions, including but not limited to job interviews. If a space analogy still works after all the complexity we've added to our "run and tumble" algorithms, I think that's probably a coincidence.
This is interesting for its breadth, but they are abstracting a huge set of gradient-climbing behaviors that I'm not sure are of the same class. Also, this has been known forever (since before I was born) in game theory / repeated games as "exploration vs exploitation".
But, it's not surprising to me that a mathematically simple, yet provably convergent algorithm appears over and over. For example, reward maximization in repeated bayesian games looks a lot like "Run and tumble", once you add this critical step (from OP):
> When a “tumble” occurs, rather than sampling a new direction from a uniformly random distribution, we sample according to the distribution of expected rewards
You may want to read the book Algorithms to Live By, also.
yep, the author is describing gradient descent over state-space - which is the driving force behind basically all adaptive systems*, though varying in both state-space complexity and descent algorithm.
This should be expected though, as neural net ML practices are a direct descendant of cybernetics, and are essentially an attempt to digitise natural adaptive control systems.
* you can reasonably analogise a single sensor/effector control loop as gradient descent against a 1d state.
Do we assume that searching in real physical space for a literal 3d smooth gradient max corresponds mathematically to maximizing the output of a planned exertion model that is internal to the computing world and probably responding itself to the neurotransmitters?
If I understand the second half of your comment correctly, then yes - optimising an n-vector fitness value over a state space (for example 2 floats from 0-1 would form a 2d cartesian state space from 0,0 to 1,1 with a continuous output value at all points), is basically the same as finding the tallest sand dune in a desert by walking around, only able to know your current elevation and the angle of the slope underfoot. Of course this is an evaluation where x and z are inputs to our fitness function and y is the output, so this is a 2d space - but this is directly equivalent to say, trying to optimise heat and humidity for maximising yield for a species of plant. You could add further variables like soil acidity or atmospheric CO2 concentration to increase the dimensionality of the state space.
I may have some of the exact terminology wrong here (whether 2in-1out is considered 2d or 3d for instance) - my interest is in cybernetics generally not gradient descent specifically - but hopefully you get the gist.
My point was that the concentration of the chemicals is not subject to the organisms internal state, and is also basically one dimensional. But my mental model predicting things is subject to the internal state of my dopamine levels, as well as very high dimensional. Pretty sure the math is a lot simpler in the first case. You will get a lot of quasi-periodic and chaotic behavior from the internal model. So bumping up the dopamine a little could easily cause period doubling or quite different, quite non-linear variations. If your gradient descent is walking a high dimensional system with non-linear dynamics, especially in a range where you are getting close to phase changes (e.g. fight or flight), it won’t be a simple little bacterium swimming to the food.
Yeah that's where you go from a linear to a nonlinear system, and why the associated discipline is called complexity science. It's still just an extension of the same concept into high dimensional space, but attractors in high dimensional space can be a lot less intuitive than 2d ones.
So the mathematics are a lot different and more complicated in the get thru a day as a human case than chemotaxis. Results of stability and convergence of a given control system that work for the linear case can easily fail for the non-linear case. And so while these straightforward maximization systems exist in the human brain, the case for this structure analogized from driving flagella to increase the chemical concentration outside might not be a universal model for human cognition overall.
Well, basically all the author did was rediscover reinforcement learning, but with a bit of additional heuristics in how to do epsilon-greedy exploration. I bet 50/50 that the algorithm will work with some tuning and training time, but I doubt what the author said can be extended to anything much more complex than those these simple OpenAI Gym experiments.
The problem I have with their argument is a lot more philosophical: it all boils down to the validity of the reward hypothesis (http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhy...), and how we even define “intelligent” behavior. I’ve written another comment somewhere in this thread for more detail.
I don't think the author rediscovered reinforcement learning -- they just didn't use that phrase or cite much of the prior work there. But they do mention a Bellman-style decayed sum of rewards, and they use the terms 'state' and 'action' and associated notation which follow the conventions in in RL, so I'm pretty sure they're aware of that field.
Correct. I'm very familiar with RL, but I didn't see how referring to RL literature fit in to my particular argument (and I'm not intimately familiar with that literature). That said, I'd suspect there are fruitful connections to make there, especially considering that this model completely glosses over learning, which is a pretty crucial element...
The K&A paper does mention the connection with RL: "Here, we consider the function of the circuit from the point of view of navigation, a connection anticipated by early work on TD learning (Montague et al, 1995)" (specifically, Montague et al found this connection when building a model of bee foraging).
If this can be turned into mechanistic theory, then implement a version of it that can solve CartPole (as an easy first test). Otherwise, it's just hand waving and conjecture.
If you want to compare humans moving through decision space to bacteria tumbling through physical space I suppose it's one way to think about how humans act but honestly that's a lot of text for a trivial kind of system that you can apply to almost anything. You can probably look at cars in traffic like this as well.
It's so vague and general of an idea that it's almost a truism. I don't think it provides anything meaningful in regards to figuring out the actual complexity of human behavior both individually or collectively.
And I think this kind of over-generalization that is a little bit vapid has been a trend because there's simply not been a lot of theoretical progress when it comes to the functioning of the human mind.
When people use this kind of thinking to apply it to artificial intelligence I always think, why not just ditch intelligence and build a physics simulator? Clearly intelligence is somewhere in the subspace of simulating the physical world, so why bother? It doesn't add much to understanding anything in particular.
If you make it to the psychology section, that's where the explanatory power becomes apparent (or at least is speculated on). Though, yes, it is a lot of text to get there (I'm relieved that some people made it that far).
If you do make it that far, there is a payoff, though. This "trivial kind of system" isn't merely an analogy but the actual algorithm that we use, and the "implementation details" of this algorithm manifest in unintuitive ways that have a lot to say about how psychology is understood in general. Though, given your skepticism, more research will likely have to be done before this is satisfactorily proven to you.
This breakthrough (which I'd say is primarily K&A's breakthrough that I merely am speculating on top of), is actually a great example of theoretical progress. I typically consider myself an anti-reductionist, but this is a lesson in how studying simple systems (in this case, chemotaxis), can lead to better understanding of complex systems (because, well, it turns out at least in this case that these complex systems are actually the same simple systems at the top level—the complexity gets added in "layers" much like the evolution of brains).
In the bacterial example, each individual tumble chooses a new direction at random, and the effectiveness of the algorithm depends on the smoothness of the chemical density function in the 3D space that the bacteria is swimming around in.
In his human examples, all the "tumbles" are conscious decisions made by the person in response to their circumstances.
If there was some human equivalent of the "tumble", it would have to be something that was change just for the sake of change. Like: "I'm bored and/or frustrated, so I am going to do something stupidly different".
So the master algorithm matches a somewhat rational behavior, from time to time. But humans wouldn't be really rational beings.
The seemingly fixed by algorithm human standard behavior will ultimately, eventually, lead to irrational behavior, by not fully able of "matching" actions with objective really.
Just like a trying to use knife (the human algorithm), as a hammer (the objective reality would be the nail).
I am pretty sure this dopamine algorithm was part of the "ten equations that rule the world" book. And it is much more clearly explained here. Worth reading the book though :-)
this is categorical thinking.
behavioral psychology, molecular genetics, behavioral genetics, ethology, endocrinology, etc. will all provide their own "demonstrated answer" to the same question, even when they will be contradictory to each other.
> Why are some people willing to free solo climb icy mountains despite the danger?
> Why are some people willing to have fewer children in order to pursue career goals?
> Why are most billionaire tech CEOs interested in space travel and extraterrestrials?
> There are no doubt potential evolutionary explanations for each of these phenomena.
No doubt?
> But, from a purely evolutionary standpoint, these phenomena are at least (a) not obviously explained or (b) seemingly irrational.
Correct.
Many phenomena have an evolutionary explanation. However some phenomena do not - random mutations happen and things pop into existence without much connection to mating, survival etc. Trying to shoehorn evolutionary narratives onto them is not letting the evidence fit the theory, but instead trying to get facts to fit some preexisting theory.
Some traits just pop up due to random mutations, and trying to find an evolutionary mating/survival/etc. narrative for each of them does not make sense.