Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Examples are good and on point, the conclusion is not. When he tries to frame it all in some grand political/quasi-philosophical manner, it becomes outright wrong and stupid, but I won't argue with that part, because it won't be useful to anyone.

What I want to point out is that nothing he says should be attributed specifically to "machine learning". Machine learning is a set of techniques to make inferences from data automatically, but there is no implicit restriction on what the inferences should be. So machine learning is not "conservative" — almost all popular applications of it are. There is no inherent reason why an ML-algorithm should suggest the most similar videos to the ones you watched recently. The same way you can use (say) association learning to find most common item sets, you can use it to find the least common item sets with the given item, and recommend them instead. Or anything in between. But application designers usually choose the less creative option to recommend (understandably so) stuff similar to what you already got.

Sometimes it's ok: if the most popular thing to ask somebody to sit on nowadays is "my face" it's only logical to advice that, I see nothing wrong with this application. But many internet shops indeed could benefit from considering what a user has already bought (from this very shop), because it isn't likely he will want to buy a second similar, but different TV anytime soon. Or, when recommending a movie, you could try to optimize for something different than "the most popular stuff watched by people that watched a lot of stuff you did watch" — which is a "safe" thing to recommend, but at the same time not really interesting. Of course, finding another sensible approach is a lot of work, but it doesn't mean there isn't one: maybe it could be "a movie with unusually high score given by somebody, who also highly rated a number of movies you rated higher than the average".



The point is that ML today is based on pattern recognition and memorizing a stationary distribution.

This stationary distribution is the source of the conservativeness and central to algorithms that we call "machine learning". ML always tries to replicate the distribution when it makes infererences, so it is fundamentally biaed against changes that deviate from that distribution

The Future and the Past are structurally the same thing in these models! They are "missing" but re-creatable links.

AI is the broader term, but in pop culture AI and ML are very much synonymous.


Well... no, not really. It is kinda hard to discuss in general, because it depends so much on the details: the application and the algorithms in question. But there is nothing inherently conservative about ML algorithms.

I see why you assume that "stationary distribution is the source of the conservativeness", so maybe I should clarify this moment. It is kind of true in the most general sense: sure, when querying the stationary distribution we can only ever ask how things are in a timeless universe. How anything new can be obtained this way? The problem is, that if we are this general, then the word "conservativeness" loses any meaning, since everything in the [deterministic] Universe can be framed as a question of "how things are", everything is conservative, nothing new can be obtained anywhere, ever.

And we don't even need to get this general for the word "conservativeness" to lose practical sense. When you ask another human for an advice, all he ever does is, in essence, pattern recognition and querying his internal database of "how things generally are to the best of his knowledge". Yet you don't call every human advice ever "conservative": only the kind of advice that recommends the safest, most popular thing, thing that everybody likes, pretty much ignoring the nuance of your personal taste. In fact, even then, you call it "conservative" only if you can notice that, which means that the recommendation isn't new for you personally (and by that criteria alone most humans would lose to a very simple, absolutely currently implementable music recommendation algorithm, since they probably know much lesser number of "artists similar to what you like" than Spotify knows: the only thing Spotify has to do to win is not to recommend the most popular choice almost every time).

One more thing. I could probably convey to you that assuming "ML = conservativeness" is wrong much faster by invoking the example of reinforcement learning, since it is sort of intuitive: there is the obvious existence of "time" in that, you can imagine a "dialogue" where it adapts to what user wants using his feedback, etc. It is easy to see how it could behave "not conservatively". I intentionally avoided doing that, since it can lead to the false impression that RL is somehow different and less conservative than other algorithms. On the contrary, the point I'm trying to make is that every algorithm, even the purest form of memorizing the stationary distribution (like Naïve Bayes) is not inherently conservative. It all depends on what exactly you ask (i.e. how you represent inputs and outputs) and how you query the distribution you have (e.g. how much variability you allow, but not only that).

So, when you see the application that uses ML algorithm and behaves "conservatively", it isn't because of the ML algorithm, it is because of the application: it asks the ML algorithm wrong questions.


I felt the same way about this essay’s conclusion. It’s like he took all of the substance out of Ted Kaczynski’s arguments and repackaged it as a diatribe against ML. Except it’s not convincing in the slightest.


Thank you. “Conservativeness” is an inductive bias that is simple, often helpful, and more often used, but not fundamental.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: