Thanks for reading and the response!

One of the points I'm trying to make is that taste and elegance fundamentally stem from an internalized heuristic -- which at it's core is memorization.

I understand the connotation of "memorization" evokes an image of blindly memorizing without connecting, but isn't the tastefully developed expertise just memorization of a better heuristic?

I don't think I can agree, as an extremely creative person with extremely bad memory - to a point where I pretty much never memorize anything, whether intentionally or by accident.

What I find instead, is that by just processing novel information, especially if I focus on analysing it, my brain internalizes insights and builds model of that type of thing, allowing me to either imperfectly reconstruct what I've seen, or to come up with an infinite array of permutations, extrapolations, etc which is where the real ideas come from.

Further, ideas crucially revolve not around just the information itself, but the "feel" for what role they play in the whole, how well they do it, in what way they're notable, etc.

In fact I'd straight up claim that memorization is antithetical to creativity - a perfect ML autoencoder or GAN would just regurgitate the training data. Creativity comes from generalisation while memorisation is analogous to overfitting.

A million times this.. I also am extremely creative and in fact I think the MOST creative people are really bad at intentional memoration, but are good at seeing patterns.

I feel like often the reason a creative person is hyper creative is they haven't memorized things so they are trying to rebuild information all the time in their heads from very sparse details.

This creates the transformative and relational combinations of information that a person memorizing can't see because it is created from a lack of organized specific information rather than a bounty of it.

I think we have a problem of semantics here. Your notion of "the brain internalizes insights" is very close to what the author means as memorizing patterns. They even gäbe a few examples where they started with rote memorizations, which were not that useful at first, but eventually a pattern, an insight if you will, emerged.

I would describe myself exactly the same way as you, and I've always been that way (noticed it at first in school where I would take forever to hand in the memorisation half of an exam but finish the analytical half in record time.)

I recommend giving spaced repetition a serious go. It doesn't cost much and you might be surprised how far it takes even someone like you. It completely changed how I view the role of memorisation in analytical work.

Strictly speaking, someone like you does not need to memorise things because you can always derive them from more fundamental principles. But being able to do that, while a blessing, is also a crutch.

Reasoning from first principles every time is slow compared to pulling out the right relationship for the problem at hand right away.

I'm this same way -- very strong semantic memory, astoundingly bad episodic memory. I've been getting into spaced repetition and I was curious if you have ideas on what to memorize via SRS?

I don't really have things in my job or intellectual pursuits that clearly lend themselves to flashcard style memorization, so I've been doing SRS on sort of quasi-useless things to help 1) get in the habit of SRS and 2) build more "scaffolding" in my memory so when I want to SRS useful things it'll be easier.

(By (2) I mean I'm memorizing some historical dates/facts because I found early in using SRS that I'd traverse known facts like a graph, so simply having more known facts would make it easier to add and remember new ones -- most things I'll learn will have a date affiliated with them, so figure anchoring a bunch of dates in my memory won't be totally useless)

I forgot to reply to this and don't want to write a long answer now in case you won't see it. If you're still interested, shoot an email to hn@xkqr.org and I'll give more details. Sorry for taking so long!

Anything you “know” is because you have memorized it. It has nothing to do with either effort, or consciousness.

Memory is the basis of knowledge.

Which is why the thesis here is boring/less useful. “All colors come from memorization” is also accurate. “All thought comes from memorization”. At that point, you’re factually accurate but saying little of use.

If you’re trying to teach creativity, what do you make people memorize? The author even points out: some cultures are great at memorizing and bad at innovation and vice versa. That’s interesting to talk about. “Try-hards use spreadsheets to be funnier” is…sad?

Yes! Creativity often happens when you try to reconstruct something you failed to memorize, but succeed at making something else.

"What I find instead, is that by just processing novel information, especially if I focus on analysing it, my brain internalizes insights and builds model of that type of thing"

Sorry, but this is memoization.

I feel like every reply making a point against memorisation would benefit from having their definition of what is memorization, because every single one of those replies sound like they're still implicitly describing some sort of memorization as the better way

I feel like this is about the difference between rote/explicit memorization and organic/implicit/tacit memorization, for a lack of better words. I suspect the former could narrow/restrict your understanding because it may be constrained/limited by the vocabulary/definition itself.

Exactly, everyone here is just describing different forms of memorization

I think perhaps there's a confusion of "memorization" with "rote memorization". The word "rote" connotes flashcards and dull drills, but memorization by itself, to me at least, is more like "a focused attempt at internalizing information", in whatever way that means to a person, as opposed to just ingesting it or letting it wash over you/osmosis.

But that's just my interpretation of the terms. I don't know what the "official" meanings are.

What do you call it when you remember things so you can repeat them but you can't generalize? E.g. if you learn a poem or phrase in a foreign language, but can't reuse the words in different contexts? Or being able to recite a rule, but not automatically applying it?

Is there a word for this?

Similarly, we should have a word for knowing how to reuse something in a different context, but not recall its origin or its canonical portrayal. Being able to apply a rule, without being able to recite it.

Do you think there's one word which means both of these things, which are opposites, as I've stated them?

> What do you call it when you remember things so you can repeat them but you can't generalize? E.g. if you learn a poem or phrase in a foreign language, but can't reuse the words in different contexts? Or being able to recite a rule, but not automatically applying it?

"Rote memorization"

Rote means learning by repetition.

And "rote memorization" is a compound term that means what you were asking for. It's one of those things you can't get the exact meaning of by just looking at the components.

Transfer learning.

Transferring skills from one context to another is surprisingly hard to do, but not impossible. AFAIK, contexts must be similar to each other for transfer to take place.

Is it? If I don't remember any of the detail, but just the general "feel" of the concept.. is that memorization?

I think I would call it internalization instead of memorization. People memorize equations not knowing what the variables are, others internalize the concepts of what is trying to be calculated.

Modeling is not memorization. It's more generic and can't allow you to reproduce the memorized information, only describe its underlying structure.

If I cannot recall the information or even that I've come across it unprompted, is it really? Because that's my norm, and I still retain insights from that, that are then applicable across topics.

Typo. Funny typo given we're on hn.

That's true if you broaden the definition of "memorization" to cover all learning, but "learning is necessary for creativity" would not be a particularly interesting thesis.

Expertise is the result of learning from past experience, both in developing an internal intuition for what you're doing and in having past patterns to draw upon. To the extent that experts have simple easily verbalizable heuristics, these are largely post-hoc attempts at explaining their intuition rather than an accurate reflection of how they make decisions.

And, in fact, experts can't even always do that: it is perfectly possible for experts to make good decisions without being consciously aware of why they are making them, and explaining how to make good decisions is a separate skill from being able to make them in the first place. The book I mentioned has a memorable story about a firefighter who thought he had precognition after pulling his team out of a dangerous situation without any specific indicator of the danger, but I figure a more common example is experts saying they did something because it was the "obvious" or "clean" or "better" way to do it and getting a bit flustered when pushed further.

We can see this in action pretty clearly if we look at advice for, say, writing. There is a lot of advice from good writers but just memorizing and blindly following this advice is actively counterproductive. Advice you can memorize fundamentally must lack nuance and context. We can see this clearly because so many different pieces of writing advice contradict each other and because good writers do not follow any of those suggestions with any consistency.

The same definitely applies to programming, which is why we have both "don't repeat yourself" and "you ain't going to need it", and why new programmers trying to apply either rule (or both!) to a codebase inevitably create a mess. What I've found with programming advice is that most suggestions are either actively wrong or too vague to be useful. (By the time you've learned enough about programming to be able to follow the vague advice, you don't need it very much!)

This happened about 20 years ago when they were trying to automate recognizing cancer cells. They showed photos to experienced diagnosticians and asked 'What features do you look for?' They couldn't articulate what they were seeing.

The concept is called 'tacit knowledge'.

Why are you attached to the word "memorization" here? Certainly taste comes from experience and learning. Maybe you could argue that all learning is an oblique and imperfect form of memorization—but why argue that at all?

The only reason I can see is if you think memorization could be a shortcut to good taste, which it can't. Acquiring good taste requires broad experience—more information than you can possibly remember—such that you retain a suite of sophisticated intuitions. Cutting that information down to something that can be memorized would require you to (1) already have the intuitions you're seeking to acquire, and (2) be able to express them all in plain English, which, as far as I know, cannot be done. No painter has ever expressed their aesthetic in such a way that a student could memorize that expression and then have the same creative sensibilities as the original painter.

Ultimately, there's no substitute for the process of simply consuming lots of art while paying close attention to what you like about it.

>One of the points I'm trying to make is that taste and elegance fundamentally stem from an internalized heuristic -- which at it's core is memorization.

seems to me there is a relatively big inductive gap there, you believe that there is an internalized heuristic and at its core is memorization, you may even have some evidence that this internalized heuristic has strongly informed your development of taste, but it is pretty difficult to make an argument that is the case for all people.

Aside from that I would say that "internalized heuristic with memorization as the core" puts everything on nurture and no input of nature - which I am pretty much in the camp of combinations of nature and nurture creating the person - of which taste must surely be a big component.

Schmidhuber reached your conclusions first.

Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes

https://arxiv.org/abs/0812.4360

not the parent poster but I think I agree with your perspective here. The alternative is that some individuals' taste or sense of aesthetics is somehow innate and unmoored from the statistics of the things they experience. There may be something to this, but for most practical purposes I would agree with your point.

Another alternative is that taste is something you can only learn through experience and mentorship, where memorizing simple rules and heuristics is not sufficient. Taste is an example of tacit knowledge[1].

[1]: https://en.wikipedia.org/wiki/Tacit_knowledge

Perhaps this is where I disagree -- I believe while difficult, all tacit knowledge can be made explicit, but is just hard to do so

This may be because I'm not good at picking up on social cues, so had to learn things more consciously

But ofc I could also be wrong and maybe there are things the subconscious can learn that the conscious cannot

There’s another argument though that some taste is genetically programmed, like our affinity for campfires or sweeping views. Those don’t seem to be learned as they seem to be entirely cross cultural and innate. Those aren’t examples of art of course but make the point that some sense of aesthetics may not be learned.

also a question - if you have better long term or short term memory how does that affect taste? How does it affect creativity, if all of these things are essentially memorization you would have to assume that people were more creative and had better taste the greater their ability to memorize things, which in the case of taste especially seems slightly absurd.

In the case of creativity it may be easier to make an argument - but surely you can find people who seem more creative with less ability to memorize.

> Memorization in programming gives us architecture astronauts and design-pattern soup rather than elegant code.

Elegance is probably orthogonal to creativity, and likely follows from some kind of minimization principle, like minimum program length. You are effectively distilling the "essence" of something from all of the noise.

Creativity seems different, more like novelty, and creativity following some kind of remix of memorized elements + some randomization seems very plausible.

You can create something novel but not elegant, and something elegant but not novel, and you can distill an elegant version of something novel that your or someone else created and that's the best of all creations.

I completely disagree with your assertion that "...rote memorization is neither necessary nor sufficient to internalize concepts."

I would recommend reading the book Moonwalking with Einstein. There is a lot of discussion there on how memory is linked directly to creativity, and to understanding concepts deeply.

---

A choice passage:

"...If the essence of creativity is linking disparate facts and ideas, then the more facility you have making associations, and the more facts and ideas you have at your disposal, the better you'll be at coming up with new ideas. As Buzan likes to point out, Mnemosyne, the goddess of memory, was the mother of the Muses.

The notion that memory and creativity are two sides of the same coin sounds counterintuitive. Remembering and creativity seem like opposite, not complementary, processes. But the idea that they are one and the same is actually quite old, and was once even taken for granted. The Latin root 'inventio' is the basis of two words in our modern English vocabulary: inventory and invention. And to a mind trained in the art of memory, those two ideas were closely linked. Invention was a product of inventorying. Where do new ideas come from if not some alchemical blending of old ideas? In order to invent, one first needed a proper inventory, a bank of existing ideas to draw on. Not just an inventory, but an indexed inventory. One needed a way of finding just the right piece of information at just the right moment.

This is what the art of memory was ultimately most useful for. It was not merely a tool for recording but also a tool of invention and composition. "The realization that composing depended on a wellfurnished and securely available memory formed the basis of rhetorical education in antiquity," writes Mary Carruthers. Brains were as organized as modern filing cabinets, with important facts, quotations, and ideas stuffed into neat mnemonic cubbyholes, where they would never go missing, and where they could be recombined and strung together on the fly. The goal of training one's memory was to develop the capacity to leap from topic to topic and make new connections between old ideas. "As an art, memory was most importantly associated in the Middles Ages with composition, not simply with retention," argues Carruthers. "Those who practiced the crafts of memory used them---as all crafts are used---to make new things: prayers, meditations, sermons, pictures, hymns, stories, and poems." ..."

Great book, motivated me to then read The Art of Memory by Frances Yates.

Although I'd say traditional mnemonic devices like memory palaces are basically linear information storage and recall devices. This can create issues in building a flexible web of information, because loci or the order of the path can become dependencies and you can run out of unique spots in a given space, leading to memory interference.

Even spaced repetition methods (e.g. Anki) tend towards fragmentation of micro-ideas. Its perfect for terms, languages, and simple one question -> one answer ideas.

I've found a hybrid method of images, nested loci and spaced repetition to be most useful, because its flexible over time, and preserves relationships of ideas.

(Context: I co-founded a SaaS in this space: www.sticky.study)

You are very correct in my experience since mnemonics backfired on me that way. It was like my brain constricted on those but recall was good in limited situation.

Thanks for sharing your alternative. I like that you’ve included your references for each component of your method. That might help as many people as your product. I’ll look into it sometime.

Great passage -- this is exactly what I was trying to get at, though they've described it with much more eloquence and historical backing.

Have never heard of this book but adding to my list now!

Didn't Einstein say that don't memorise what you can look up? E.g. nothing nowadays since we have the Internet.

The internet has so much information we often can't actually find what we're looking for once we get past a certain surface level understanding. Or people don't want to pay to host it anymore. Our people disagree with it and it's taken down.

I don't disagree with Einstein, but I wonder what he would say with the modern internet at his disposal. Maybe the same?

All of us over here memorizing words to speak instead of looking up each word each time...

The book Make it Stick taught me that this Don’t Cramp My Style With Your Boring Rote Learning (Man) attitude is prevalent in teaching. At least American. They argue that it is wrong for the same reason that the author does.

But saying this to a programming crowd must be the most futile thing. At least instrumentalists have to rote train their muscle memory. That lowest bar has to be passed, even if it’s just three chords.

But the article isn’t about programming creativity though. It is a general concept. But if honing in on the mythical lone-genius activity (geniuses never practice in a structured way) helps you win an argument then so be it.

I think it is more of becoming fluent with primitives that can be composed in versatile ways. I can see how that can be poorly understood as memorization.

The main implication is that if what you are “memorizing” is not easily composable, then you won’t be able to apply them broadly or creatively.

However, I disagree with the author on what creativity is, although his definition is one experience of a creative inspiration.

Interesting perspective. I do agree that there are people out there who develop a distinct "taste", but I can't tell if this refers to a "style", an emergent property of multiple "habits", etc? I've always wondered how one develops their "taste".

Also, would you consider a subconscious habit "memory"? What's the difference between the two?

Counterargument: the alphabet.

> One of the reasons people make fun of the author's approach to creativity is that systematic memorization fundamentally can't teach taste—so the systematic approach reeks of awkward, try-hard, low-brow, tasteless art.

Well... can you think of an artist who didn't have a deep knowledge of their art-form before they pushed it forward? Three that jump out for me, in no particular order, are Picasso, Borges and Jack White. After all, great artists steal.

There's a big difference between "artist who didn't have a deep knowledge of their art-form" and "artist who didn't follow an explicit system to memorize a bunch of rules to make their art".

Wait, why do you think Picasso didn't have deep knowledge? He studied both at the School of Fine Arts in Barcelona & the Royal Academy of San Fernando in Madrid, for ~5 years before moving to Paris.

Borges was incredibly talented, but it's worth keeping in mind his dad was a writer too.

Good art very much relies on being exposed to lots of other good art first. I don't know that rote memorization is the best way to achieve that, but you definitely need that exposure.

Sorry, I must have expressed myself badly. I'm picking examples of people I think did/do have deep knowledge of their chosen mediums.

I don't think it's possible to have "good taste" without exposure to lots of examples, because I believe taste it culturally bound. Whether you do it explicity via a system, or on a more ad hoc basis, I think most artists need it.

It might be interesting to look at film, where the process is compressed into a couple of generations. I don't know it it will support my argument or not.

Ah, misread you then, thanks for clarifying.

I don't think film will look very different here - early film work was very much informed by theatrical tastes at the time, and then started to diverge as people figured out what else they could say in the language of film.

Fundamentally, all art exists in a cultural context. If you've ever taken an art history course, you've been hit over the head with that info a few times ;) And that means furthering/changing taste in a given field means being aware enough of the existing rules to deliberately choose which ones you're breaking, and why.

There are some (very few) artists who didn't have a formal grounding, but I'd argue that even they were steeped enough in cultural context to be informed by it. Even famous autodidacts like Grandma Moses did develop a love for art based on being exposed to a bunch of it.

(Fully recognizing that it's a somewhat tautological argument because it's kind of impossible to grow up in a society without being somewhat exposed to its predominant art forms)

Picasso was a hack. People often cite his "early masterpieces", but those pieces are pretty mid in the context of 19th c. painting.

I think that argument is that these artists did not memorized rules or previous pictures and then applied them. They did put a lot of effort into learning, but that is different claim. If you define "memorization" as "any learning of anything", then the word is kind of useless.

Pollock is often regarded as pushing painting forward, for example

Thanks, I don't know enough about him - does that support my hypothesis or tear it down?

Pollock was obsessed with creating an art style that had no basis in any other style, something truly "original." He felt the abstract style he created fit that aim.

That being said, in some ways you could say that the splatter paintings he's known so well for are in fact influenced by all the art he studied and discarded along the way. They were definitely influenced by the principles of artistic design he learned, even if they looked different from what people were used to.

In my opinion, your hypothesis is supported, though maybe in a bit of a roundabout way.

> Well, okay, but rote memorization is neither necessary nor sufficient to internalize concepts.

Of course it is. It's how every human child learns initially. By rote memorization. How does a toddler learn how to say mama? By constantly hearing and repeating it. How does a kid learn their ABCs? Rote memorization is the basis of all memory.

> Memorization in programming gives us architecture astronauts and design-pattern soup rather than elegant code.

Dumbest thing I've ever read. You write programs well by doing and remembering. Same with writing. Memorization is the necessary component to programming well. In other words, you program well by remembering elegant code.

> For me, at least, trying to memorize something without context

After the basics, most memorization is contextual.

> At the same time, memorization has a real cost: it takes time and it's frightfully dull.

Oh dear. Something isn't fun all the time. What a childish worldview. It's more fun to eat candy and drink soda than eating 'dull'. It's more fun to sit and watch youtube than to workout.

> Sometimes a bit of memorization is unavoidable, but I've found that to be relatively rare.

Relatively rare? In order to be competent in anything, you have to memorize lots. You can't write a good essay without having memorized much of the material. Trying reading a book where you have to constantly look up definitions of words because you lack the vocabulary. Try having a conversation with someone who has to constantly look up words because he lacks the vocabulary. Try having code review with someone who doesn't remember anything about their code.

> Otherwise, my time is generally better spent on some sort of practice in context.

Why? Because it helps you remember?

To the idiot ( probably OP ) who downvoted, try coding without having 'memorized' the keyboard. The anti-intellectual, anti-hard work, anti-memorization agenda pushed by some 'people' online bears looking into.

Strong points, but insults and emotion aren't how we do it on HN.