Thanks for this reminder about Pieter Hintjens.
I have been greatly influenced by his writing, including his use of Alice & Bob, in "a protocol for dying" !
So witty, so clever - such a tragic loss.
"“I think we’re at the end of the era where it’s gonna be these giant models, and we’ll make them better in other ways,” Altman said.
He sees size as a false measurement of model quality and compares it to the chip speed races we used to see. “I think there’s been way too much focus on parameter count, maybe parameter count will trend up for sure. But this reminds me a lot of the gigahertz race in chips in the 1990s and 2000s, where everybody was trying to point to a big number,” Altman said.
As he points out, today we have much more powerful chips running our iPhones, yet we have no idea for the most part how fast they are, only that they do the job well. “I think it’s important that what we keep the focus on is rapidly increasing capability. And if there’s some reason that parameter count should decrease over time, or we should have multiple models working together, each of which are smaller, we would do that. What we want to deliver to the world is the most capable and useful and safe models. We are not here to jerk ourselves off about parameter count,” he said."
May be a little OT but came across this technique called MABs multi armed bandits , related to , finding optimal ways to decide among a sequence of actions and measuring their utility
Has applications in a wide variety of fields : optimal design of clinical trials, public policy decision making.
apparently has some relation to the work abhijit banerjee & esther duflo, the nobel prize winning economist couple have done on design of experiments, to measure impact of developmental interventions
Not OT at all, IMO. As it happens, the linked book actually touches on MAB problems explicitly, albeit briefly. And as the book points out, you can formulate a MAB problem as specific form of Markov Decision Process, and MDP's are referenced extensively in the book.
I find MAB's incredibly interesting, and for pretty much the same reason(s) I find the rest of this stuff interesting.
