There is also the entire bend where these are taking off in a way that doesn't make sense.
It makes perfect sense. Just not to a lot of HN readers.
The average HN reader is approaching this from a perspective of "I am a professional programmer who might occasionally dabble in scientific computing, and therefore I hate this thing because it's not a professional programmer's tool designed by and for professional programmers according to the best practices of professional programmers".
The people who are actually using notebooks, meanwhile, are not professional programmers. They're scientists who increasingly have to do programming as part of their science. And notebooks are a godsend for them. We don't need to drag them all the way into our world; we need to pay attention to what they actually want, need, and find useful, and accept that it's going to differ from what we want, need, and find useful.
They're scientists who increasingly have to do programming as part of their science. And notebooks are a godsend for them
2/3 of scientific research cannot be reproduced by other scientists. But tell us more about why scientists should ignore best practices from other fields.
Because I don't have four years to get something done, that doesn't do what I want when I finally get it, if it even works at all, and that I can't fix myself.
Okay, that was extreme, and if you think I was talking about programming, it's because you have a guilty conscience. ;-) It actually applies to all interesting fields -- programming, engineering, management, classical music composition, etc. Those fields don't even know what their best practices are, and acknowledge that things take too long and can't be managed. No manager would say: "Our programmers have best practices, so the work will be done next week." Why should scientists have such faith?
Meanwhile, do you trust Maxwell's Equations, Darwinian evolution, quantum mechanics, etc.? How did we establish the physical constants to mostly better than 8 digits of precision? Science has somehow figured out how to make progress despite the messy business of research.
For me, it's not that I "have" to do programming, but that physical science has been computation driven since before the 1940s. Programming is how I think and work. With apologies to Richelieu, "programming is too important to be left to the programmers."
> Those fields don't even know what their best practices are
"Best practices" are a chimera. The issue at hand isn't about what is "best", but whether or not a software engineer's "good enough" practices are more likely to achieve science's goals than a graduate student's "good enough" practices.
It's also disingenuous to claim that classical music composition doesn't have "best practices" when the field of music theory exists as an explicit manifestation of "best practices" in music. Having gone to a school with a conservatory, I also believe that I know several individuals who would would disagree with your mindset regarding how the creative process can't be managed. Indeed, if creativity, as it relates to musical composition, couldn't be managed most orchestras would be brimming with anger at the number of commissions that weren't finished on time for the concert, and most Hollywood studios and Broadway shows would screech to a halt.
Show me the reproducible research in programming about the merits of different type systems (murky at best). Or of different approaches to testing. Or software architecture. Or... well, most of the stuff day-to-day working programmers actually do. There are barely even attempts at rigor in most of our practices, let alone the kind of reviewed and reproduced results we demand from the sciences.
I run my unit and integration tests with every build, and they reproducibly pass if my code is working. If you have code, it doesn't take much to make it able to run again and get the same result, and it's frustrating to see Jupyter users mess it up.
It makes perfect sense. Just not to a lot of HN readers.
The average HN reader is approaching this from a perspective of "I am a professional programmer who might occasionally dabble in scientific computing, and therefore I hate this thing because it's not a professional programmer's tool designed by and for professional programmers according to the best practices of professional programmers".
The people who are actually using notebooks, meanwhile, are not professional programmers. They're scientists who increasingly have to do programming as part of their science. And notebooks are a godsend for them. We don't need to drag them all the way into our world; we need to pay attention to what they actually want, need, and find useful, and accept that it's going to differ from what we want, need, and find useful.