I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.
I always hated notebooks personally as I found them clunky and inefficient, but never wanted to yuck someone else’s yum.
Then I started mentoring a junior staff member who worked on another project. The lead of that project was a physicist who wrote primarily in Jupyter notebooks. Like, thousands and thousands of lines of code. This junior staff member spent like 90% of her time confused and copying-and-pasting between notebooks. She had no idea what a virtual environment was and her go-to solution for solving import errors was to nuke and re-download the repo and re-run the setup script, or create a new notebook from scratch containing all the prerequisite user functions.
Was top 10 most horrifying things I’ve ever seen. I advised her immediately to stop using notebooks for development and sent her a few Python tutorials. Luckily though she just left the project and got on a better one instead.
This is why spaghetti code is so prevalent in scientific community. We are not really train (especially for non engineering research projects) to become programmers and we do programming just as a way to do research.
I still rely heavily on principles acquired from courses on Software Architecture and PL studies as a part of my CS degree - and I can definitely see a difference in how people organise their code with same tenure but no schooling.
Remember back university, we used to do course projects (for science/engineering course) and at the end we would have to write a report. The report would typically include, among other stuff, two things.
(1) The calculational methods we used - could either be a set of mathematical equations or a description of the algorithms. (2) The results of evaluating these equations/algorithms for different parameter values. Usually some graphs, and some discussion of their meaning.
A Jupyter notebook is designed to replicate that process but make it easier because the figures are produced by code right there. Personally, all my notebooks include a discussion in the markdown cells what I am doing, and why. It includes discussions of the code. And directly from the code, some graphs or numbers, with a discussion attached.
With the script workflow, I would have two different files. One with the code, and one with the results pasted in. It's annoying when my primary goal is to develop and test the algorithms under discussions. Best thing is, if done right, my work is completely replicable. Just run the notebook again.
Just because some people misuse the tool doesn't mean the tool isn't useful.
Sometimes you want to quickly iterate on a portion of code doing some sort of analysis or tweaking a plot. If your data pipeline takes a while to run then rerunning the whole script is really awful. Notebooks make it easy to cache rerun and tweak chunks of the code.
You can't embed graphs in a script, and plotting is an important part of systematic research. Also, it is easier to have an obvious sequential set of experiments over months in a notebook rather than a bunch of scripts. It's the same reason scientists use lab notebooks to keep track of things rather than just a bunch of loose papers.
Yes, I have a lab notebook and I work in a lab. However I use folder structures to organize my scripts and as for graph outputs I save them as files. In my line of work I generate hundreds of graphs for data analysis verification and a notebook isn't set up to handle things like that as far as I can tell
You are asking what the advantage of a notebook is. Can you truly not see a job / workflow in which inline plotting would be advantageous? I find that very hard to believe.
My workflow normally starts in the shell (SQL, R/rush/ggplot csvtk, etc), moves to Jupyter (F# in .NET Interactive, with some SQL and R/ggplot inlined), and then to a Makefile when I’m ready to make some deliverables.
I’ve got an ever growing tool set of F# data related functions that I’ve moved to a personal lib in my dotfiles and use in scripts, notebooks, etc.
There’s also a hybrid version that I like: Write proper scripts (which are easier to e.g. push to HPC servers or deploy later), and explore them using something like Quarto (neé RMarkdown, renamed now that it supports Python).
Then you still get a digital lab notebook that ties together scripts, plots, and documentation, but the scripts remain usable standalone.
Another advantage is when you have very slow code, you can use cells as caches, essentially without having to worry about serialization to disk. This often makes it much easier to interactively explore/develop downstream methods without needing to re-run earlier upstream dependencies.
This is especially useful with large datasets. Even if serialization is straightforward, if you have enough data (or the data is remotely hosted), loading it might take anywhere from 2s to multiple minutes, and even 2s is enough to get you out of the flow if you are working rapidly and want quick feedback.
You can rerun something halfway down a script without re-running the whole thing.
If your script requires loading 12+ GB of ml models into a gpu before running anything at all this is the difference between a few seconds and a minute to see a change also if the output isn't text you can see the image or chart result inline to that code.
I think the workflow improvement happens but it's not because notebooks allow you to do something that you can't do otherwise. They just improve ergonomics.
For example, there are a lot of cases my team uses notebooks for proofs of concept where we make a large expensive call to load a large chunk of data, slice a small piece of it, iteratively try to reprocess the piece until you get the reprocessing to occur the desired way, validate it reprocessed correctly, and then extend the reprocessing the the rest of the data set. That can all be done after only making 1 expensive call. Further more, if the last cell evaluation fails, it just resets you back to the line before and you can retry it.
Can you do this with a script? Absolutely. You can write a script to download the data, and a script to process the data, and sub scripts for the individual steps. But that's not the path of least resistance; the path of least resistance involves you having to edit a piece and recompile everything and reset the entry point. Avoiding really makes it easier to brute force to the desired state ASAP.
you would have to re-run script from the beginning - this is not productive in scientific experiments, where you need to re-run certain parts of your code and tune/change parameters, try different things.
if your calculation is long running you would not be as productive as could be in notebooks
On the contrary, it’s a backport of the (IMO) most useful notebook feature to work also for regular scripts. It’s been ported to most code editors by now (VSCode “Interactive Python”, Vim “hydrogen”, Emacs “code-cells.el”, Sublime “Send Code” extension, Spyder has it built-in, etc.), so a lot of people have found that feature useful.
In contrast to Jupyter, you’re still working with plain text files and not JSON, and you don’t end up saving the cached data in the same file as the script.
In contrast to Quarto, Jupytext, etc., this is still just a code file and not a MarkDown file with code blocks. Not all editors have fully working “go to definition” etc. in MarkDown code blocks, and in any case, many people need a standalone script that can be placed in an HPC job queue after initial local testing is done.
A REPL, by its name, is a very narrow version of the broader paradigm of interactive computer programming environments. But Notebooks are not REPLs, unless you use REPL to mean "interactive programming environment" and not REPL. Notebooks are much broader than a REPL! In a notebook, you can go back and edit and run individual lines of the notebook without re-running the whole notebook from the start and without re-computing everything that depends on what you just edited. Behavior like this makes it super hard to track the actual state, and super easy to lose track of how things are how they are. That's pretty terrible!
"In a notebook, you can go back and edit and run individual lines of the notebook without re-running the whole notebook from the start and without re-computing everything that depends on what you just edited."
Isn't this a standard on REPLs as well? You can select the code you wish to run, and press Ctrl+Enter or what ever. I must admit, I've programmed Python for about 10 years in Spyder and VS Code now, but I haven't used notebooks at any point. Just either ad-hoc scripts or actual source files.
My definition of a "notebook" is an ad-hoc script, split into individual "cells" which are typically run as a whole. On my workflow, I just select the code I wish to run. Sometimes it is one expression, one line, 100 lines or 1000 lines depending what I've changed on the script.
> Isn't this a standard on REPLs as well? You can select the code you wish to run, and press Ctrl+Enter or what ever.
Not usually, no. Type `python` at the command prompt - what you get is a REPL. Type `clisp` at the command prompt, or `wish`, or `psql`, or `perl` or even `bash` - those are al REPLs.
Very different to a program that presents an editor, and then lets the user selectively choose which lines/expressions in that editor to run next. For example, type `emacs somefile.sql` in the command prompt. The application that opens is most definitely not a READ-EVAL-PRINT-LOOP.
Why would adding fancy select or cut-and-paste features to a REPL make it not a REPL? Selectively choosing which lines to run is just a convenience to let you not have to type the whole line or set of lines again, it doesn’t really change the base interaction with the interpreter.
> Why would adding fancy select or cut-and-paste features to a REPL make it not a REPL?
For the same reason that adding (working) wings to a car makes it not a car anymore.[1]
I mean, to my mind, when something is satisfying a different primary use-case, then that thing is a different thing.
I'm sure there's some fuzziness in the distinction between "This is a REPL/car and this is a Notebook/plane".
Usually it's very easy to see the distinction - the REPL is waiting for the next command and the next command only while the notebook takes whatever input you give it, determines whether it got a command or content, and reacts appropriately.
[1] Tons of examples, TBH. I don't refer to my computer as my calculator, even though the computer does everything a fancy calculator can do. People don't call motorcycles 'bicycles', even though the motorcycle can go anywhere that a legal bicycle can go. More telling is how people don't call their computer monitor 'TV' and don't call the TV a 'Monitor' even when the same actual item is used for both (i.e. I repurposed an old monitor as a small-screen netflix-box, and now an item that used to be called 'monitor' by wife and kids is called 'TV' by wife and kids).
a flying car with wings is still a car. that's the whole point of it. it can drive you to the airport and drive like like a car. I don't care what you call your computer, it can still do math. people who have their TV hooked up to their computer would more readily refer to it as a monitor. idk, I just think REPLs are kinda shit for interacting with the present state of a kernel (as Jupyter calls them). Jupyters better, but still kinda shit because it could automatically infer the important variables in scope and keep a watch list like a debugger does. And then suggest things to do with them since it's a repl and not an IDE. but the thing is fundamentally they're Read Edit Print Loop interfaces to the computer and its current working state.
Ugh this is a gish gallop of broken straw man analogies. Being able to select and evaluate a single line in a notebook is nothing like adding wings to a car. Fundamentally, selecting a line to evaluate is no different from typing that line again. It’s a shortcut and nothing more, the interaction is still read-eval-print. Note REPL doesn’t even refer to where the input comes from, the point is simply that it handles your input, processes it, displays the result, and then waits for you for more input. This is as opposed to executing a file where the interpreter exits once the execution is completed, and the return value is not automatically printed.
Jupyter Notebook absolutely is a REPL, see my sibling comment above for the WP link describing it as such. It waits for input, then evals the input, the prints the return value, and then loops.
I’m not sure what distinction you’re trying to make. Maybe you can give some examples of notebooks that are not REPLs, since some of them definitely are. For example, Wikipedia says Jupyter Notebook is a REPL. The bare Python REPL (and the command line REPLs in any language, for that matter) has the exact same issue with tracking state, because what you describe is a problem with all REPLs, and all notebooks that are REPLs. That isn’t generally a serious problem with command line REPLs, because those REPLs aren’t meant or used for large system programming, they’re for trying small experiments. The parent article is pure opinion and seems a bit confused about the idea of using the right tool for the job, because command line REPLs and notebooks both have their place, as do IDE projects with lots of files.
“A Jupyter Notebook application is a browser-based REPL containing an ordered list of input/output cells which can contain code, text (using Github Flavored Markdown), mathematics, plots and rich media.”
I still have no idea. I assumed it was a paper notebook, but then Excel is brought in, so now I think it's an Excel feature. In either case I have no idea what their are on about.
Agreed, even something like "coding" notebooks if brandname usage is a concern would help. It took more time than I'd like to admit to understand that they weren't talking about physical notebooks...
One of those complaints is due to an unfortunate implementation choice.
Out-of-date cells happen because Jupyter works like a buggy makefile that doesn't reliably rebuild dependencies, forcing you to run "make clean" when anything weird happens. There are better build systems.
Observable notebooks will automatically rerun cells that changed, like a spreadsheet. It works nicely for calculations that aren't too heavy, but it might not be what you want for a heavy batch job.
Their newer tool, Observable Framework, works more like a regular build system. You can still have it automatically build when you save a file in your editor.
A second complaint, that it's browser based, is basically an editor preference. You can open Jupyter notebooks in VS Code if you prefer.
Notebooks are a diluted form of the Lisp/Smalltalk REPL-based development experience, with some features from the reactive-spreadsheet world. Especially for those of us in the business of producing numbers, 'real code' with a fixed set of tests isn't a better way but a necessary evil, a black box that we can't really trust. Building a calculation piece by piece, in a notebook, trying out variations along the way? That's how you get a feel for the calculation, connect with it at a spiritual level.
I wonder why CL and Smalltalk haven’t beat Python. Is is the languages or just unawareness? The workflow just make more sense there with better updates propagation and state saving.
The Clojure/JVM statistics/scientific computing/now tensor math packages just never got as good as Python, and in Smalltalk they were a non-starter. R is an awkward language, but it's repl-first, has a lot of Lisp's metaprogramming (done in very ad-hoc ways), and Smalltalk's serializable image model -- so a lot of exploratory/experimental statistical methods research happens there, and then gradually makes its way to Python when it needs to be stabilized for production.
F# based .Net would've made a fine math-centric environment, but it's a language with a high initial barrier, and though .Net community has been making a decent effort at porting over a lot of NumPy/SciPy, it's not fully caught up after many years.
We have Julia now, it's jitted, multicore/GPU friendly, and has interesting REPL innovations, and yes, serializable state (though, ugh, ligatures). But every time I reach for it, it's like, oh no, yet another thing I want to call is in the Python ecosystem. Especially all the modern deep learning/tensor stuff, where the assumption is Python's speed and the GIL don't matter because you're just gluing together GPU calls.
> The Clojure/JVM statistics/scientific computing/now tensor math packages just never got as good as Python, and in Smalltalk they were a non-starter.
There are undergoing efforts to improve numerics performance on Smalltalk. Pharo, for example, uses JIT, and already beats pure Python. But numerics in Python is mostly outsourced to fast modules written in C/Fortran. So people are in process of making Pharo's counterpart to NumPy/SciPy (PolyMath) leverage BLAS/LAPACK integration as well. See:
In terms of NumPy port, for more involved implementations you might be better off using built-in numeric primitives together with upcoming Tensor<T>(kind of like ndarray on steroids) and TensorPrimitives (BLAS).
It's absolutely hilarious that the author opens with an image of Socrates.
Socrates, as you recall, famously argued that writing was a detriment to thinking. The parallel that notebooks are a sign of lazy thought does not go unnoticed.
Some of the worst code I've seen in my life lives in Jupyter notebooks. But that's fine, it's meant to be throwaway code. The problem is a lot of places/people do not use it as such.
In general anything written by nonprofessional programmers in pursuit of some other goal is terrible code by professional programmer standards.
On the other hand, it accomplishes a goal other than getting a programmer paid. So in one important sense is objectively better than probably 80% of the code I've seen people paid to write, no matter how nicely it was constructed.
This is the crux of a lot of the anti-notebook rhetoric.
I spend a good deal of time covertly teaching software engineering best practices to folks who claim they don't want to be "software engineers", yet they are in front of Jupyter most days.
I'm a so called "scientific" programmer, and I use notebooks. I care about good programming practices, and have made an effort to improve my craftsmanship.
And I don't want to be a software engineer. I'm surrounded by SWE's in my work area, and their job doesn't really interest me. Division of labor. I'm glad somebody enjoys it.
Amen to that. The confluence of an insulator nature of organisational developer cliques, the culture of self-importance in the field, and the ability for many teams to expend material effort on things that are…at best, tangentially related to business goals (due to cheap money, and frankly, often pulling the wool over management’a eyes), has bred at least a couple of generations of developer where large contingents have a completely out of whack perspective on what’s actually important.
In one project, I had a new teammate ask me why almost our entire web backend is a single .js file mostly filled with SQL queries. He wanted to make it his personal project to refactor this into probably 20 different files, adding more layers between the handlers and the DB stuff, using a query builder, and also migrating to TS. When I asked him what's wrong with the current thing, he couldn't answer and gave up on this idea.
That backend had decent integration tests, and it took less than 5 minutes to add a small new feature. Most prod code doing comparable things where I work is worse-tested, less reliable, and at least 10X more expensive in terms of SWE-time, so I think modern programming standards are actually nonsense even though I can play along with them.
Yeah I think I got this from Sandi Metz a long time ago but it's been so valuable to me: "bad code that you never have to modify is good code."
Now when business requirements change and you're constantly sending people into that 1600 line backend function or whatever, it's likely worth it to start doing some refactors. But otherwise if I don't have a good reason to be in there I pretend I don't see it.
I agree with the quote, but this was code that we did modify frequently. It was still ok. 1600 line func is bad, but 1600 line file, well that's what editor splits are for. A lot of app backends don't do much other than shoving stuff into a DB and getting it back.
We'd edit queries or add new HTTP endpoints. Some did a little more than query, maybe hit a cache or external service. There were a few shared helpers, not more than one layer deep though.
They should put it in an excel spreadsheet. Then it would be carefully documented and be immediately retired, no possible way to outlive its usefulness.
The insidiousness of notebooks is in their uncanny resemblance to proper code. To take writing for example, a handwritten draft scrawled on coffee-stained yellow paper is more obviously a work-in-progress than a typed manuscript. But the maturity of code is not so immediately apparent.
I have masses of admiration and respect for Jeremy Howard and tried using this to build a bunch of fastapi endpoints.
I still have masses of admiration and respect for him, but no longer so much when it comes to software engineering - due to this, among other things. It seems like a great idea, but is clunky, hard to refactor and to integrate with other things, in my practice.
Wrapping production code in jupyter cruft doesn't work for me.
Or, any code, really, that doesn't stay very small and stand alone.
This is a great article. For me, a lot relates to scale.
I use notebooks occasionally for the university classes I teach. I like how I can write equations and text to explain ideas, and present interactive graphs to illustrate the ideas.
But it's quite a lot of work to set things up, and there can be a problem when components of the work take a long time.
In my research work, it is common for a calculation or a graph to take hours to days. Doing work like that in a notebook is just a non-starter. I use scripts (in various languages) along with Makefiles that "know" if I've changed my code or my data, and only rebuilds a result file or a graph when require. Almost always, I separate code that does analysis from code that creates graphical displays. And of course the text (of a paper, course notes, etc.) is written to incorporate these numerical and graphical results. The whole point is to subdivide complex tasks into smaller tasks that can be executed in an organized way.
I don't see the sense in using one tool (notebooks, say) for simple tasks and other tools (scripts, a strong editor, tmux, unix, etc.) for complex tasks. So, apart from the toy tools that I make for some teaching tasks, I stick to simple unix-style tools for the tricky stuff.
It's important to have many tools in the toolbox. Notebooks can be useful for some things, but I wouldn't want to frame a wall using an awl.
Recently, since coming across FastAI and nbdev[0], I've been moving increasingly to a more notebook-centric flow. So far it's been better particularly for the exploration aspect where I've primarily used ptpython in the past (and this isn't anywhere ML-related). I think the idea behind nbdev is pretty neat, but it pushes some practices that I'm not a fan of at all. I want to get to the point where I have a mostly complete IDE experience in the notebook so I don't have to keep switching back and forth. I have a ways to go.
I remember Jeremy from fast ai was an advocate for notebook centric flows. But I have to respectfully disagree that he is a good authority on maintainable / scalable coding practices like this. And though I may be wrong, I feel like the fast ai lib itself is a not too useful wrapper around pytorch and people would be better served just learning pytorch itself. I say this as someone who watched a year of the fast ai videos, and got really excited about ai because of the fast ai course, which I am still grateful for. But this is my current take.
I'd say the fastai library itself[0] is a pretty good example of how maintainable/scalable practices can come to life in notebook flows. There's something to be said IMO for an active project with 25.8k stars, 238 contributers, 2.7k commits, and 199 open vs 1.5k closed issues.
I'm an embedded person, so clueless on this stuff. Are notebooks really used in prod? Outside of applications where notebooks are specifically useful to the goal of the system, ie. documentation,blog posts,textbooks with interactive/data driven elements? How would this even work? Can an ipynb file be mechanized in the same way a py file can?
Jupyter notebooks can be executed roughly like scripts by papermill. You can also save a .py version of the notebook without outputs using jupytext. We use these packages together where I work to basically auto-generate the start of notebooks for exploratory work after certain batch jobs. For dashboards only used by a small number of users, we’ve found voila occasionally useful. Voila turns notebooks into dashboards or web apps basically.
You generally shouldn’t put code you want to reuse in notebooks, but we haven’t found this to be much of a problem. And >3/4 of the people who’ve worked on our team don’t have software engineering backgrounds. If you set clear expectations, you can avoid the really bad tar pits even if most of your co-workers are (non-software) engineers or scientists 0-3 years out of college.
I think the issue actually goes beyond notebooks. It's just that notebooks are the method du jour for creating hacked-together code. But regardless of the tool chosen, hacked-together code does eventually leak into production. Mine has.
It's harder to breach that barrier in embedded, because someone has to approve of getting the hardware made.
In most organizations, there's a kind of no mans land between hacked code, and professionally written software. That's because the real SWE's are always overbooked, and for good reasons -- business and career wise -- should be working on the biggest projects. This leaves software needs that are urgent for somebody but not urgent enough to rise to the top of the priority list for the software team. The somebody could be an internal or external customer.
The article and comments debating best practices in notebooks remind me of the time I hosted a website from a jupyter notebook.
Technically, I was using gradio to create a localhost webpage and then piping it through cloudflare. The website would only work when the notebook was running on the cmd line of EC2. Hey if it works, it works! Notebooks allowed me to do a 2 week project in 2 hours.
Like most things, notebooks have their pros and cons. One of the biggest adv is very rapid experimentation. Even a regular script takes a while for python interpreter to run and that time (even if only a few seconds) adds up in lack of creativity ("Bret Victor - Inventing on Principle" [1]). And if your script is loading a big database, then notebook is a no-brainer.
One of the biggest disadv of notebook is mis-ordering. You are allowed to declare a variable in cell 3 and then go and use it in cell 2. Even worse, you can declare a variable in cell 3 and then delete cell 3 and still be able to use that variable. That I believe is the biggest dis-adv of notebooks. It adds way too many subtle errors. One way to bypass this is to write everything in functions - no global vars.
I am willing to accept the issue of mis-ordering in order to get rapid experimentation. It's subjective whether you think the pros outweight the cons. I definitely think they do.
A notebook is a REPL with an inline wiki. Of course it is not intended for running production code, it’s just an R&D environment to document, share, and test ideas
>Same with notebooks, you can write NASA-production-grade software in a notebook, but most likely you won't.
One of my family members literally maintains a software platform for test bedding production NASA spacecraft at JPL and it is all on top of Jupyter notebooks.
People are literally on course to build AGI and there’s a good chance that large portions of that work will be done within Jupyter notebooks.
To suggest real work doesn’t get done in notebooks is simply asinine.
As I have embedded further into data science and machine learning teams within my career I have come to learn that a lot of the things software engineers care about around maintainability, reusability etc. are often completely antithetical to data science productivity. In many cases practitioners would consider some of the listed cons of notebooks as features rather than bugs.
There is obviously a time and a place for pulling out those practices in such settings but blindly applying software engineering standards to this kind of work is both all too common and often rather counter productive.
If notebook style dev doesn’t jive with you personally I get that, and god speed. I’ve worked on CV/ML teams with a lot of CS degree grads and they don’t always want Jupyter notebooks. However, I’ve noticed a tendency for folks to extrapolate that to blanket statements about notebooks in general. As someone who has inherited and had to clean up many jumbled/bad notebooks I’m empathetic but I would challenge folks to lean into considering more about the circumstances the previous notebook author was working in and what kinds of productivity outcomes they were responsible and prioritizing rather than casting judgement on said author or their tools of choice.
I think we need to clearly distinguish two things here, though. A lot of software engineering ritual is dedicated to shipping software - that is the part that tends to be of very remote relevance in research and scientific contexts. But there are also more fundamental principles and techniques that concern writing code, which apply regardless of the reason why you're doing that, and are beneficial in any context - in a sense that, if you follow them, you will spend less time overall writing code (and thus more time doing whatever it is you're trying to achieve with that code).
However, the problem is that those things are not trivial and have to be learned before they can be applied. And that is where the value becomes questionable - the time spent learning all that could instead be spent on learning other things. But if you already have a software engineering background, you do benefit from doing things "right" to some extent.
A notebook is just an IDE for writing scripts. There's software you can use (like Databricks and Azure Data Factory) to orchestrate them. But obviously if you are building an application they can't be the only IDE you use. A lot of the confusion in this article comes from trying to use them in a way that they're not designed to be used.
This is a very narrow minded take. Notebooks are used mostly by scientists who are just using code as a research tool. Being able to explore to learn is incredibly useful. Using them in production is a perfect bridge between different disciplines. And if you get distracted because “your browser has tabs” that’s really not the notebooks fault.
Notebooks are not IDE, so cannot be compared to ones. You can run notebook in vscode, and keep the best code practices. Experiment with code, incrementally move code chunks to external modules. Another great feature is ability to eaisly switch kernel to remote one without leaving IDE. Downside is that it can be really annoying to manipulate the code.
In the context of machine learning, notebooks are a crutch people lean on when the production environment lacks the right abstractions--isn't convenient enough. Notebooks fall under the "prototype smell" in Google's Hidden Technical Debt in Machine Learning Systems (2015).
I agree that notebooks are best reserved for exposition.
I use them too, but for data science rather than machine learning. The right abstractions include handling authenticating to the ML services, experiment tracking, hyper-parameter optimization, distributed compute, access to PII data, etc. If you're dealing with nonsensitive data that fits on a laptop you might not need these.
> Notebooks are often used as a playground to easily have access to an accelerator. In this case, the execution environment is often set up differently and with different dependencies. In this case, you make your code work in a colab, smile widely and run your experiment after on a cluster or whatever. You start crying in twenty minutes after your code crashes due to lack of dependency or versioning or anything else similar.
This is actually one of the best things about jupyter is that you can just tunnel into the computer and use the exact same env... this article is WACK
Notebooks are used for crunching data, for scientific, mathematical, or engineering purposes, and showing your work (revealing to your colleagues exactly how the data was crunched). If you're deploying them to production you get what you f'n deserve.
That said, when I'm writing code, if I want it to be good quality code then I might use org-mode and its literate programming facilities to type up my thoughts, interlineating code as necessary. This is more useful on greenfield and personal stuff, but there is a high probability that I too have undiagnosed ADHD and the use of org-mode helps to organize and clarify my thoughts before I commit them to code. It is a hallmark of the programmer personality to jump into coding without thinking things through beforehand -- or really, sometimes, at all. The management consultant Tim Bryce wrote about a programming class he took in the 70s:
"When given an assignment, the other two members of my team started coding immediately, then keyed up the source code (Yes, we were using punchcards back then), and kept compiling and debugging until they finally got a clean compile. While this was going on, I pulled out my trusty old template, worked out the logic, wrote the code, keyed it up and usually got a clean compile on my first shot. Inevitably, I beat my teammates everytime."
The use of org-mode and LP helps with these organization and planning issues... well, at least it does for me.
I have no idea what he is talking about. I use notebooks because I want to have an interactive and visual way to explore data/algorithms/phenomena. How/Why would you use a notebook "in production"? Why would you care about performance?
The article really makes me think either I or the author have a terrible misunderstanding about what notebooks are or how to use them.
If there are complaints about code quality in a jupyter notebook, chances are that someone is doing it wrong -- either the author is using it for demonstration purpose, for production or near production environment, or the person looking at others' notebooks is taking them too seriously.
Most code in jupyter notebook is bad, but that's fine, because nobody should waste time doing that unless they must reproduce the results or they are sharing the notebook with others.
Did I miss somewhere in the article where the author says what the chosen alternative is to notebooks? "I'm also constantly exploring other options and having fun" makes it sound like there isn't one.
I have an understanding that the notebook is something other than laptop, but I can not find neither the definition of the word nor any synonyms in the article.
I get that the gentle snark is the point, but I feel like a tiny shift in perspective would universally improve "take" posts like this.
Add a "why I might be wrong" section, or "why this might not apply to you". Just show me that you've thought from more than one side of the issue and I'll be more receptive to your side.
I'm not even a programmer, I'm a data scientist 1 year out of college, and I still know that notebooks should never ever be deployed to production!
The right way to view notebooks is as.. notebooks. Text and visualizations that can run online code. They are for presenting!
You would need a tremendous amount of incompetence and a lack of any code reviews or tech leadership at all to end up deploying a notebook to production.
Notebooks are primarily a _teaching_ tool, not code or an experimentation interface. I know that's not how they're used, but it's really bizarre to me that people use them for exploratory data analysis. They're great for teaching and for _documenting_ the results of exploration, but fairly bad for any type of interactive data exploration, let alone actually writing code.
Frankly, notebooks are pretty terrible for exploratory data analysis, especially in python. The main issue is that they absolutely kill interactivity. Straight up CLI based ipython is far more interactive, and often better for exploratory data analysis. In a lot of ways, pretty much any other interface is better than code blocks that aren't independent but can be run independently.
Part of it is the interface (i.e. non-linear state, code blocks, etc), which is great for teaching, but bad for exploration. However, i.m.o. even more of it is the fact that it's browser based. E.g. folks often think it's best to use matplotlib in a notebook. However, using a notebook means you get very little interactivity out of a highly interactive library and remarkably poor performance out of something that's actually _very_ interactive and quite performant on standard Tk/Qt/etc backends.
If you ditch browser based stuff, you're usually far better off. This part is mostly just me griping, but I don't understand why we've stepped back 20 years in basic usability, interactivity, and performance by making everything use a web browser (and then making that not even remotely cross platform).
The issue with CLI is it's crap for editing the code. It's rare that your data analysis will just be one-liners, it'll be some mess of steps and you want to go edit one in the middle (but not re-run the whole script). That's the first thing that the notebooks give you that you don't tend to get elsewhere.
The second thing is the presentation of the plots: it's really handy to have multiple interactive plots laid out right next to the code block that produced them.
There's nothing that requires that this is done in a web interface, but no-one seems interested in making a non-web version (IIRC there was a QT version in the very very early days of ipython notebook but it's long dead). One big thing that any replacement will need is the easy ability for the code to actually be running on a remote machine, something that's trivial for a web interface but requires active effort for a native client (and makes a lot of interactive chart work more difficult: certainly there's currently no other matplotlib backend which can do it).
I'm fairly sure a better interface than jupyter is possible for data exploration, but I haven't found one yet since there's not really anything else that gives you the two features above.
(one thing people often miss about notebooks and the kind of work that go into them is that the code itself isn't actually really hard code. It's 100% about interacting with the data and the algorithms running on it. Working in one isn't much like software engineering)
You're not editing code in the CLI. You're editing code in your preferred editor/IDE. ipython + %edit works wonders. That's one of the main reasons to avoid notebooks and stick with CLI-based or similar tools.
Jumping into the middle of a script and editing is often a bit dangerous. You usually wind up with something that can't be reproduced, as it depends on modifying state in a way that wasn't recorded. You do often need some sort of caching during exploration, for sure (it's not really feasible to wait an hour every time you want to tweak a visualization), but it's often better to break pipelines up a bit and do disk I/O rather than leaning on cells in a notebook.
I do agree on the remote client part, though. Browser-based interactivity often is a good solution for a dev instance + a local thin client. It's slower than native for a local solution, though.
> It's rare that your data analysis will just be one-liners, it'll be some mess of steps and you want to go edit one in the middle (but not re-run the whole script.
And that's why I use elpy to execute codeblocks marked by comments in IPython. VSCode supports this workflow out of the box (but you don't get the matplotlib windows out of the box ;)) - although it is very, very liberal in using screenspace...
> One big thing that any replacement will need is the easy ability for the code to actually be running on a remote machine
So have I (not FAANG, though). I've been working in scientific python for over 20 years and long before notebooks came to the python world. (Yes, pre-numpy) And I've seen notebooks abused far more than used well. The same general thing was heavily abused in Matlab/etc too. It's not only Jupyter and it's not new.
It's not that they're not a good tool, it's that they're the wrong tool for a lot of what they're used for. They're an excellent teaching and documentation tool. However, they're remarkably bad for actually doing any type of analysis or exploration. They're a great way of presenting your analysis after you complete it, but they're bad for actually doing that analysis. Write analysis code and data exploration code using the same tools you would for production code. There's nothing wrong with an edit + run + explore loop. Do that with a standard REPL like ipython + your preferred editor.
But there's a lot wrong with trying to use a notebook to write and execute code, as it:
1) encourages writing code that won't actually run later (dependent on non-linear code execution and unsaved changes made to earlier cells), and
2) is a poor environment for writing code compared to more full featured editors or IDEs, and
3) discourages interactivity and data exploration due to being browser-based (poor performance compared to running directly limits interactivity during data exploration).
Your data analysis work needs to actually run independently later. Notebooks rarely do unless they're re-made from scratch. At that point, you're better off working in a more full-featured environment for the exploration phase and then using the notebook as later documentation / communication.
I disagree with the idea that notebooks and McDonalds are somehow inherently and fundamentally bad. They fit a certain niche, and you should use them accordingly.
> If people are given a chance to be lazy, they will be (I will be for sure). Quickly hacking some stuff without properly testing? Sure! Linting? Pfff, living on the edge, you'll throw the colab away in an hour anyways
To save everyone time, I think the fundamental disagreement comes down to this. A vast chunk of the audience of notebooks is people whose brain has not been drilled on the daily about best practices and maintainable code and often don't get judged on that criteria or affected by the consequences of this. If you believe that these people are irredeemable as engineers, or more charitably that the incentives driving this cannot be fixed, then you probably believe in taking away good tools from these people because they'll misuse it, and then you'll agree with these articles. If you don't, then you won't.
> I am using code autoreload and write code in modules
>> Nice! I've done this for a while as well. This is a good use-case. However, this approach does not address some of the issues, e.g. data analysis scripts should be checked by another team member.
How come? When you write your code in a module, it gets committed and PR'd and linted and tested and reviewed just like any other code, which is the whole point of this approach. Which I think is the most reasonable and mature one.
The notebook then is mostly reserved for a very thin interaction layer optimized for play, on top of your vast iceberg of reviewed & tested & maintainable code. Including lots of prewritten convenience routines for plotting and munging to minimize business logic in the notebooks further, and not have paragraphs of matplotlib crud in hidden cells. You construct what's almost like a shell + DSL that's optimized for the domain and datasets you care about, and you can focus on investigating. If there's more significant code, it soon goes in a module again, without interrupting your ongoing flow and thought process.
As a web developer who now is increasingly coding in python.. I sympathize with this but frankly google colab is way more accessible to me than other options. Jupyter notebook is good because it provides a quick feedback loop. Google colab is needed by a beginner like me because they have taken care of all the pytorch / cuda set up. The tutorials I access online use it so it's not easy to find a different way. All the quick and easy tutorials only demonstrate using google colab (or some other similar alternative).
I messed around with paperspace for a while, but I could not get detectron2 working in it so I went back to google colab. And my ML projects, that I have ambitions to become production apps, are still toys so for now the quick and dirty notebook will have to do. But yes, I prefer to be in my element in a traditional IDE, pushing to git, tabbing between files. I certainly have > 10 years of experience doing that, but not really for ML projects if that makes sense.
Hum... I do not use Jupyter and co BUT I have my configs and various personal stuff in org-mode/org-bable witch is a kind of notebook and my desktop boot to Emacs, who not retain state but it's still a live environment and it's flexibility and user-programmability with ease are it's strongest points.
So well... Writing in org-mode makes me waste a bit of time but produce much clearer designs because I do not design in code but in natural language and if the narrative is suboptimal is immediately clear, also it proper document the code non with comments but with complete human language. The downside it's that's very time consuming and I do loose some coding helpers that can't work much in babel blocks instead of full source files.
So well, for me such interaction is the most powerful UI we have, the best human-computer interface so far, the default go-to solution for any day-to-day work. A project could be an independent entity but still the day-to-day work born and evolve like that, so I'm not agreeing much, notebooks are not quick&dirty stuff but a tool to reason and reasoning often take much more time than directly write code, if you are disciplined. On contrary might disperse you if you are in "silicon valley mode". This is probably than main point not of the tool, but of the human using it: how many are constantly in Silicon Valley mode vs in Donald Knuth mode?
I've never been a fan of notebooks, it always felt like a strange way to run my code. What's the real advantage over creating scripts? If I want my code to run somewhere I would need to notebook architecture or just turn it into a normal script anyhow
Notebooks have at least one advantage, but it's one that shouldn't really be an advantage: In a world where every big organisation rushed to adopt all the latest cloud buzzwords, we now have an awful lot of half-baked fontends for AWS and Azure that will certainly be mandated by company policy. Actually getting a computer that can access all the data you want and run in a way you want through EU law, US law, EULAs, company security policy, cloud billing policy et cetera et cetera is a nightmare.
But for some reason, no matter what nonsensical environment you're being forced to operate in, there's always a Jupyter web interface with access to exactly what you need that none of the auditors, accountants or systems administrators seem to care about. It may not be good, but it works, and won't be bricked by some inexplicable change to IAM roles or VPS settings applied by some automated policy tool you didn't even know exists.
For a notebook/repl environment, you can create any number of intermediate steps, rerun the previous step with minor modifications and check if the results are better, rinse and repeat. For jupyter notebook specifically, you can visualize data and add markdown inline which are very useful.
You won't understand it unless you are already familiar with the workflow.
Sometimes I need to run my code in small pieces for testing and evaluation, I do this with multiple smaller scripts. Data can be saved to files and this accomplishes nearly the same thing as a notebook without needing to have the notebook environment
Sure, if you don't mind the overhead (both development and processing time) of loading/saving state to disk (about 10-30 minutes for a lot of my data). In notebooks you don't have to think about it since it's just the objects in memory (and indeed they don't make it easy to think about it, which is a reasonable criticism, but I don't currently know of a framework or system which gives you the advatages of both).
Whenever I work with large datasets I use a small subset of the overall data to do testing while I build the pipeline, this avoids long run times and allows for quick iteration while I get things set up to run against the full dataset
of course, running a script from start to finish is better when it's fast, but if your code is slow to run, each cell acts like an in memory cache of your previous work.
There's so many other ways to achieve the same effect though. I would argue that serializing the result of some expensive computation explicitly is even just better, as it can have a meaningful name and metadata.
Just serialising and deserialising the results can be expensive though: it's quite for jupyter notebooks to have >64GB of data loaded. Even with high-end SSDs that's a fairly significant overhead.
I've seen them used really effectively for debugging and incident response & documentation in elixir and erlang. You can connect to a running node in a controlled way, repl around the live environment, and dump potentially interesting things out to the notebook as you build your understanding of what's going on.
I wasn't too active in ops stuff at that job, but as a dev it was so awesome for them to hand me off a notebook that was basically an in-progress investigation, including the entire path of exploration they took to narrow it down and dead ends along the way. You can't ask for a better bugfix ticket.
Now I guess a lot of that depends BEAM particulars, and doesn't apply to jupyter or whatever. But it's at least an illustration that notebooks can uncover novel workflows for certain tasks.
It’s disappointing to have someone’s misguided hot take get lots of attention. There’s nothing inherently wrong with Notebooks, just like there’s nothing inherently wrong with a command line REPL. They are tools. Sure, neither is a great choice for coding a million-line application, and yet the REPL is the better choice for one-liners, and the notebook is the better choice for a few plots or developing a 10-line algorithm. I’d rather read something thoughtful about when and how and why to migrate from a notebook to something better. Making that easier, without judgement, is going to do a whole lot more good than attempting to shame people with a bad analogy will do. Anyway, McDonalds has scaled larger than any other food chain in the world, so the title is accidentally paying a huge compliment to Notebooks.
I try to avoid Jupyter related things. Every time I encounter them, it seems worse. Point in case, an open issue from 2017 requesting "copy/paste" be (re)added to the right-click context menu, still open.
