I had my mind lightly blown watching someone using Squeak Smalltalk to interactively prototype something or other.
They purposefully called a method which did NOT exist, used the debugger to partially unwind the #doesNotUnderstand stack, implemented the method, which then returned the expected result as if the method had existed all along. Benefits of super late binding I guess.
Whether or not they can support patching changes into a live system. You can get really good debuggers but lacking that capability you won't approach what Smalltalk and Common Lisp offer. And if it's non-standard (say you do implement it for C with a specific compiler and debugger, but no one else does it or all do it a different way) then you won't win any mindshare outside of that target platform.
Clojure user here, and I often don't know the details behind this, can you explain? When trying to explain the repl it's. easy for folks to say "okay, don't I have that too?" but yea, really the updating of values from your editor into this running application 'universe' seems like it should just work for every language, so what is special about lisp that enables it?
It's possible to do with probably every language out there, it's just not always as practical. Common Lisp, Erlang, and other languages were (in part) designed around interactivity with the running system. In many ways, at execution time, programs in those languages are actually running on a separate operating system hosted by your main OS. This environment is tailored specifically to monitoring, updating, and extending the running program. Compared to C, which has no such infrastructure baked in. At run time, it is whatever you told it to be at compile time (modulo dynamic libraries that may have been loaded). To manipulate C programs at runtime you have to attach a debugger, which does let you do quite a bit. But you're also stepping outside the C language to do it. Versus Common Lisp, Clojure, Erlang, Elixir, and others which present an interface to the running system that is the same (or very nearly the same) as the language itself, and even permits definition of new functions within that interface (you aren't going to write a C function in a C debugger and load it in place of another C function in your running program).
There are a couple reasons and ways to use that approach.
Unlike batch compiled languages, Smalltalk and Common Lisp (and some others) will throw an exception/signal an error/whatever that indicates that a function doesn't exist. This means that you can insert these calls and, unless they actually occur, the rest of your program runs just fine. In most batch compiled languages you'd at least need to insert a dummy method/function (in C# you'll see NotImplementedException being thrown in these).
When you encounter the exception, in languages that support it, you can write the code at that time and then let the system resume, or tell the debugger to return a value. Maybe you don't know how to implement that function, but you know the kind of thing it should return and, for some reason, you don't want to leave function floating out there that just returns a dummy value permanently ((defun random () 4))).
If you use a language like Smalltalk, there's also a tendency to write very short methods. You could use this approach to create your program in a more top-down fashion (versus bottom-up) by doing something like this (I don't know Smalltalk so this is lisp):
(defun main() (factorial 10))
Debugger is brought up because factorial doesn't exist.
(defun factorial (n)
(fact-aux n 1)) ;; not really needed in Common Lisp, but just to illustrate
Again the debugger is brought up so we define the next function:
(defun fact-aux (n acc)
(if (<= n 0) acc (fact-aux (1- n) (* n acc)))
If you already have a design in mind (sketched out on paper perhaps), you could use something like this pretty effectively. Not sure I would though.
The REPL is still something that I haven't quite mastered, but what I enjoy doing right now is working in files and using a REPL connection to evaluate expressions within the file. It feels like a happy medium between "do everything at an (ephemeral) REPL prompt" and "do everything in a file, save, and reevaluate the file."
My weapon of choice right now is vim-iced[0], which is a batteries-included VIM plugin that, in addition to providing a REPL and keybindings to evaluate expressions under the cursor, also allows me to auto-format my files on save.
> The REPL is still something that I haven't quite mastered, but what I enjoy doing right now is working in files and using a REPL connection to evaluate expressions within the file.
For me that is the optimal way of doing things.
I want to restructure stuff, make multiple versions of functions, move them in and out of comment blocks or just leave code there for REPL use.
When I first saw someone writing Clojure like this on a video I was thinking: "This is the way I want to write code." I think this is the best of both worlds.
> I want to restructure stuff, make multiple versions of functions, move them in and out of comment blocks or just leave code there for REPL use.
I think programming at the REPL requires a very different mindset from how a lot of us learned initially. The thought of leaving commented code in our codebase is one of those things we're told not to do, but there's a great snippet from one of Stuart Halloway's talks where he mentions "Rich Comments" (pun intended I think).
Rich is apparently known for leaving REPL code inside comment blocks specifically for testing purposes, and as a sort of transcript of the work that was done. [0]
Its a great approach, and one that I've adopted, but it seems wrong at first glance when taken out of the context of a REPL.
Wow, this just blew my mind. I usually keep a bunch of ; commented out lines at the bottom of my file so I can still do M-x cider-eval-buffer when I load up for the first time, but this makes so much more sense without having to comment and comment things all the time.
Yeah, something I've seen become common in the Clojure ecosystem is for files to have a `(comment ...)` form at the bottom where forms which are commonly needed at the REPL when working with the file (e.g. to initialize some data or a database connection) are kept around. Editors like IntelliJ with Cursive have commands like "evaluate sexp before cursor" which lets you easy use them, and since they're all commented out they don't effect the normal execution of the code.
This lets you have many versions of code in one file (if that's a sensible thing for you, very useful when exploring a new library or concept though), and in the end you can aggregate it all into one source file for sharing with others:
If you are doing Clojure development in emacs CIDER makes things super easy. Loading files or buffers into a repl as well as eval region and sexp (there are a ton more features but those are the main ones I use)
I don't have much to add to this discussion except to say that after years of repl-driven development in Clojure I've switched to Ruby as my primary language, and the iteration cycle is just really slow. I feel like I've traded in my fighter jet for a commuter car.
How would you contrast developing with the Clojure REPL compared to irb? I've found irb to be pretty useful for some quick and dirty tasks, albeit not the same level as Clojure (or most Lisps, really).
My experience with IRB has been so-so. I've found it fine for experimenting with small functions as I develop them, poking at interfaces, etc. But it's really nowhere near the experience I had with Clojure and a repl-integrated programming environment. I've only been using Ruby on Rails for a little under a year, so I have some more learning to do, but I'm not hopeful about the productivity story.
Repl-driven development in Clojure with Emacs and CIDER [0]:
You develop a namespace or two at a time and you're looking at a handful of files in your editor. You can move between function definitions, modify them, and re-evaluate them as easily as moving the cursor and pressing a key. The whole editor becomes a playground: every piece of code you see can be changed and evaluated as fast as you can think. You can execute a web request from cURL or the browser, and your Clojure repl is hooked into the running program. So you can capture real inputs, store them in temporary vars, play them back, change them on the fly, etc.
The closest thing you can get in irb, is to use Rails' `reload!` after making changes to your files. I don't remember if irb supports this kind of stuff out of the box, perhaps good old `require` works.
Differences:
- Clojure has namespaces, so you can reload just one unit rather than the whole code - I guess load/require would be closest
- You can also just evaluate a single form (e.g. a function definition) leaving everything else intact
- if you're using something like Component for state management, then you never restart the repl, just stop the system, refresh your repl state and start the system again - closest thing to 'refresh on request' from Rails' dev mode or PHP, but without the nasty surprises
The drawback is that restarting a repl is a slow process, but it's something you do very rarely. Some folks keep their repl process around for weeks.
Clojure[Script] is worth learning just to experience a REPL-driven development environment.
Some editors and IDEs support it better than others, though, notably Emacs and Atom. Cursive (for IntelliJ) is pretty good, and Calva (for VS Code) is promising, but they lag behind in polish and features when I tried them. For example, I don’t think Cursive has inline output next to the expression you’re evaluating like Proto-repl for Atom does, and once you’ve tried that you don’t want to live without it.
I don’t write much Clojure these days but miss the REPL-driven experience. I recently discovered Quokka and Wallaby for JavaScript/TypeScript:
They offer a great experience that beats console.logging your way to success or step-through debugging by far.
Just highlight the expression you’re interested in and get inline output (in the paid version of Quokka) right in your editor, including evolution of values in for loops and the values of data retrieved from fetch requests. It has greatly shortened the feedback loop for me and both plugins were much easier to set up and configure than Clojure’s REPL integrations.
There is a non-virtuous (vicious not the right term) cycle at play.
Many problems are only solvable with compiled languages, for performance, correctness, or other non-functional considerations. Many people are first taught on compiled languages, and bring those approaches to dynamic languages. The popular dynamic languages, in their standard/common toolchains, do not have robust repl support (looking at you Python and Javascript). The absence of a robust repl workflow means people who are exposed to it in those languages aren't really impressed by it. Really too bad.
Compiled doesn't have to mean batch compiled. Most modern Common Lisp systems are compiled, but you can compile smaller units (down to individual functions).
The problem with the way we're taught (or I was taught, circa 2000), was that it was still in the batch-oriented mode: "Gather all your source files, and run them all through the compiler together." "But I only changed one thing." "That's fine, `make` will do the right thing for you." "But I also have to close the program." "Well, yeah, there are ways around that but we won't teach you, maybe your future coworkers or employers will."
Clojure is a compiled language btw. It's just compiled on the fly. When you're evaluating new functions at the repl, Clojure is generating Java bytecode and dynamically loading it.
Thank you Alex, I know, "compiled" in my post was lazy shorthand for something like "low abstraction, batch compiled" which in the context of the non-virtuous cycle limiting the reach of the repl/conversational workflow also touches on the historical and pedogogical legacy of people being smarter than the machines and needing brains to work at low levels of abstraction. Now that's rapidly, though unevenly, becoming a myth.
Your work over the years on Clojure is so deeply appreciated and my post was also to some degree coming from a place of lament that more people don't know about it and benefit from it. Thank you for it.
In fact Mesa had one on XDE, which was improved in Mesa/Cedar, because the team wanted to cater their environment to the Lisp and Smalltalk developers at Xerox.
Wirth built on top of this when he designed Oberon, where the OS only supports dynamic linking, and every public exports from a module can be used on Oberon's REPL as a command.
Although many people aren't aware of this, Eclipse always had a crude REPL via Java Worksheets, which allowed to execute Java expressions and saving those sessions on the workspace. This trances back to Smalltalk transcripts and Visual Age for Smalltalk heritage.
Eiffel also offers a REPL like experience in Eiffel Studio via the MELT VM.
.NET had Linqpad for quite some years, until Visual Studio got interactive C# and F# shells, and immediate debug mode goes back to the VB days of yore.
So there are plenty of options with compiled languages, but like with graphical debuggers, teaching is lacking on the availability of such tooling.
Yes, when I work in python, I do that too. It certainly works ok.
The place I would start from a critique perspective is around lifecycle. Python has encouraged carelessness and overwork at module load and init time, and there aren't common protocols (that I have seen) that involve standing up a graph of objects across an application after a single module reload. So when you reload a single module and the names in the module now point to different data and code elements, all the downstream dependencies on those names have to change as well, and Python doesn't have patterns for organizing and managing at that level.
This also impacts runtime introspection, which is a key benefit of a repl oriented language. One implication is that there is not at all a pattern of "repling into" a running python application to poke around and see what is going on. To most python programmers this is a foreign concept.
Related others involve code style, and implicit vs explicit ways of referring to and moving data around, which also impacts runtime whole system introspection.
Python has all the pieces to have a very nice repl-first programming model, but lacks patterns for key parts of that- explicitly standing up the graph of live objects, and using a repl to interact with a live system with live objects.
In short, the Python process has to restart to take changes in, and you loose your objects in the REPL. The Lisp process doesn't and is built to (lazily) update your objects to reflect structure changes (such as if you deleted a class slot). I particularly like the Lisp way for unit tests or the web.
I feel like working in the Unix CLI is absolutely REPL programming with the same workflow, using pipes and filters and small programs.
1. mess around with some program, echoing things to it and looking at outputs
2. up-arrow in shell, edit previous experiment
3. repeat, adding more onto pipeline, until happy
4. save the good version to a script, eg echo "!!" > foo
5. add a #! line and chmod +x to promote it to a command
The Unix CLI is most definitely a REPL. But it is a system-level REPL. So instead of running functions you run programs. You can't interactively develop a program in the UNIX CLI. Especially not in Python, JS, Ruby, etc. You can try to establish a REPL like flow in the CLI, by using a watcher like entr and doing something like $ cat test-input.txt | python ./my-script.py but the granularity of this approach is too coarse for the programming _language_. It is the ideal granularity for orchestrating shell scripts and CLI programs.
I think it clicks quickly for some people but not others (me). I don't get to use Clojure for work currently (.NET shop) but I really like it. In my spare time I remake a lot of things I do at work to learn Clojure using real world scenarios.
I've read this guide several times over the past few years, watched a bunch of the videos and I still find myself stoping/starting the repl very often because it feels difficult to keep the current state of the environment in my head, or something isn't updated as I expected, there's an error I can't decipher in the REPL that the compiler gives better info on. I feel I need to restart constantly, it's still my main workflow. I'm not sure what it is that's not clicking for me, but I keep trying. Either way, I'm no worse off than with my C# workflow.
If there was a way to visualise everything that's loaded in the repl, it would be great. As in, let me easily & visually explore the world of the repl right now.
Ditto with other langs, I just started with Spring Boot & Kotlin & geez, Spring annotations are poison chalice material. Googling how to debug spring leads you to 'just turn on debug logs' which dont tell you 1/1000 of what you'd need to know to debug annotation issues.
As you might be aware, Kotlin has a nice repl for kotlin script files.
If there was a way to visualise everything that's loaded in the repl, it would be great. yeah, I don't think that exist
I think it's maybe related to 2 things, especially for web development.
1. Unless you strictly design functions that are decoupled from any type of UI (such as your back-end "business logic"), it's not straight forward to do REPL driven development on its own.
2. If you're going to go down the path of using the REPL, you might be inclined to do TDD instead because now your efforts are saved to repeatable tests rather than a short lived REPL.
The thing those people miss is that "tested in the REPL" means tested once, and tests are there to be rerun many times, basically any time something is changed. You write your first test in the REPL, and then you transfer those tests to a proper test suite so you can reevaluate them all quickly. And because you ran it in the REPL, you also have a way to easily grab what should be the correct value.
It's not just functional languages. It's dynamic languages that tend to have a better REPL experience, because nearly the entire language can be used within the REPL, and nearly everything written in the REPL can be placed into conventional source files and used there without alteration.
Additionally, both languages you listed (Java and C#) are batch-oriented. They assume you're compiling large units of code (at least a whole source file, which in Java means the entire class and all its methods and constructors). Translating that to a REPL is challenging. You now have to leave class definitions open (which is not the normal Java behavior, a class cannot be altered without jumping through hoops outside its initial definition). So you make those hoops available via special REPL-only constructs, your REPL is no longer a Java REPL, it's a partial-Java REPL with additional things.
My primary REPL experiences have been with these languages: Python, Common Lisp, Erlang, Matlab.
2 of those are functional (as a primary part of their design, Common Lisp is multi-paradigm). Python has some functional elements, but is an OO/imperative language. And Matlab (at least when I used it) was definitely an imperative language.
But the thing they all have in common is that nearly every line of code could be moved between the REPL and a source file without alteration (Erlang is deficient here in that you can't define functions in the same way in the REPL as in the source code, and I think Matlab had the same issue but it's been 15 years).
Another thing that languages with a good REPL tend to have is a general trend towards being expression-oriented rather than statement oriented. What's the return value of an if statement in Java or C#? It doesn't have one, how would you interact with that in a REPL, then? In Common Lisp, if expressions return a value (the result of evaluating either the true or the false branch).
Because it's no different than running tests in a language with a fast code/run cycle. Clojure by necessity, needs to have a REPL, without one it would be unusable. The reason: Clojure startup time is pretty bad.
I do a lot of scientific computing, mostly in Julia, and having a REPL at my disposal is extremely useful. There's a great VIM plugin [1] that enables you to send code from the editor to a terminal to be executed, e.g. a tmux session which has a REPL open. It can be coupled with other nice plugins such as vim-ipython-cell [2] and vim-julia-cell [3].
A REPL can also be excellent for operations work and ad-hoc tasks. I use it a lot when I need to extract some data from a REST API. Can create a toolbox of methods over time for making calls, and add new ones in an ad-hoc fashion, then use those to script larger tasks. Exploration, analysis, minor ETL tasks, debugging state of deployed systems, etc. etc. can all be enchanced by a REPL.
yea I'm an idiot and need to get my hands dirty with response types so I'm often just building up lots of top level `(def body ..)` variables that will eventually be moved into a single function.
But just messing around in my editor connected to a repl lets me really breakdown a task while I'm figuring it out.
Can someone knowledgeable explain how are lisp REPLs different from Python / Ruby REPLs? What is the differentiating point of REPL driven development?
Is it that you interactively test your functions and add them to source file once they work? Or is it that with editor integration, you can load functions to REPL as they are defined? Am I missing something obvious?
I've answered similar questions several times over the past few years, but I don't mind repeating myself. It offers me a glimmer of hope that my preferred way of working may not fade away, after all.
Consider the standard Common Lisp generic function UPDATE-INSTANCE-FOR-REDEFINED-CLASS (http://clhs.lisp.se/Body/f_upda_1.htm). It reinitializes an object when Lisp detects that the object's class definition has changed.
Ask yourself this: who would call such a function? Why would anyone ever invent it? Not only did someone invent it, a committee of some of the world's smartest and most experienced Lisp programmers wrote it into the ANSI standard for the language. What were they up to?
UPDATE-INSTANCE-FOR-REDEFINED-CLASS is not a weird anomaly; it's part of a carefully-considered set of features and protocols designed to support a specific style of programming. The Lisp runtime calls it for you automatically when it touches an object whose class definition has changed.
If you've defined a method specialized for it, then Lisp executes that method to rebuild the touched instance as if it had originally been instantiated from the class's new definition, and then your program goes its merry way. If you didn't specialize UPDATE-INSTANCE-FOR-REDEFINED-CLASS for this case, then the Lisp drops you into a breakloop.
A breakloop is an interactive repl with full access to all of the runtime's memory and all of the language's features, including visibility into the whole call stack that landed you in the breakloop. You can wander up and down the call stack, inspect anything in the runtime, edit bindings, redefine types and functions, and resume execution either at the point of control where the breakloop started, or at any other point for which the breakloop exposes a restart.
UPDATE-INSTANCE-FOR-REDEFINED-CLASS is not the weird fever dream of a confused eccentric. It's part of a purposeful system design intended to support a style of programming in which you build a program by interacting with a live runtime and teach it, interaction-by-interaction, how to be the program you want, while it runs.
It's a particular example of a general approach to programming best exemplified by these old systems. That general approach is the answer to your question: "Can someone knowledgeable explain how are lisp REPLs different from Python / Ruby REPLs? What is the differentiating point of REPL driven development?"
The differentiating point is that the entire language and system is thoughtfully designed from the ground up with the assumption that you're going to be changing your work in progress while it runs, and that you should be able to change absolutely anything about it as it runs and have a reasonable expectation that it will continue to work while you do it.
I like to call this style of programming "programming as teaching", and distinguish it from the much more widespread "programming as carpentry", in which the programmer is, metaphorically speaking, at a workbench banging together artifacts and assembling them to see how they turn out.
To be clear, I do not claim that the teaching approach is objectively better than the carpentry approach. I claim only that I, personally, am happier and measurably more productive using the teaching approach. I know that some other programmers report the same thing, and I suspect that if the teaching style of programming were more widely known, then there would be more programmers who prefer it.
There are several sibling comments that assert that any language can be made to support repl-driven programming, or that offer various languages and systems as examples of repl-driven programming. I'm sure that's all true, for some relatively restricted version of repl-driven programming, but the gold standard in repl-driven programming is programming as teaching in the style of old-fashioned Lisp and Smalltalk systems. These old systems offer amenities that the younger alternatives touted here do not match. I want more people to be aware of what they're missing.
Starting in the 1980s, I grew accustomed to systems that could start from cold in about a second, presenting to me a complete interactive development environment with all tools preloaded and ready to work, with the whole dynamic environment of my work in progress in the same state it was in the last time I was working with it. Moreover, I was accustomed to being able to take a single file from one machine to another to reproduce that same whole working environment equally quickly and easily on the new machine.
I could save the entire dynamic state of the running system to an image file, a serialized version of the running system's memory. I could later start up the system with that image file and be exactly where I was when I saved the image, right down to the positions and contents of all the open windows. I could save an image showing some bug or some strange behavior and give it to a colleague so that they could see it, too, and interact with the restored dynamic state to debug it.
I enjoyed comprehensive whole-system reflection that enabled me to view and edit absolutely everything in the running system while it ran. I could inspect absolutely everything, including the development environment and all its tools, interactively change any variable or field value, redefine any type or function, and continue to work with the changed system without stopping and restarting. (Obviously, if I made a bad change I might break the system, but remember, I could kill it and get back to where I started in a second or so).
I could start some process running--perhaps a 3D animation in a game, or a discrete-event simulation, or whatever--and change any values or definitions I liked to see what changed in the running process, without stopping the process to rebuild. For example, I could tell a rotating copper cube to become a glass icosahedron and reasonably expect to see my changes immediately reflected in the running program. This property is invaluable not only in games, simulations, and any kind of work with a visual-design component, but also in any kind of exploratory programming, where you're constructing data structures and evaluating expressions interactively to test your ideas.
Similarly, I could build some speculative data structure to explore an idea, and define some functions to operate on it. I could evaluate those expressions to see their results or to change the example structure. I could inspect the structure interactively and edit it in place if I think something different would work better. If I think a problem is caused by some structure or value in it, I could use the inspector to change it and see. If I thought one my my functions was doing something I didn't expect, I could insert a call to break, to activate a repl from inside the function call that would enable me to inspect and edit the data structure, redefine the function, and continue from there.
Anything the development system could do, I could do by typing an expression into the repl. As an example, nowadays you can still rebuild the whole Clozure Common Lisp environment from the ground up by typing (rebuild-ccl :full t).
The point is not that I would want to rebuld my Lisp from the repl all the time. The point is that the repl doesn't impose any aribtrary boundaries on what I can do. If the language and development environment can do it, I can do it from the repl. This is one of the properties that distinguishes the whole-system interactive design of these old tools from the more limited repls offered by newer ones. In pretty much every repl I've used other than old-style Lisps and Smalltalks I'm all the time stumbling over things you can't do from the repl.
I mentioned breakloops above. Their absence in younger languages and tools seem to me like some sort of sin, like we're tragically abandoning some of the best lessons of the past. Few newer development systems have them, but they're incredibly useful--at least if the language runtime is designed to properly support interactive programming.
A breakloop is a repl with all of the same affordances of the normal repl, but extended with all of the dynamic state of the control path that invoked the breakloop. If an error or an intentional call to break triggers a breakloop somewhere deep in a stack of recursive function calls, you get a repl that can see every frame of that stack, and every variable and value lexically accessible from it. You can browse all of that whole, change values, and redefine functions and types. You can resume execution at your leisure, and any changes you made in the breakloop will be visible in the resumed computation just as if that's how things were originally.
Proper breakloops don't just improve error messages; they replace them wholesale with an entire species of programming that lays the whole dynamic state of the system out on the table for you to examine and modify while the program continues to run.
Moreover, everything I just described about breakloops can also be automated. These old systems provide not only interactive tools for rummaging through the dynamic state of a suspended computation, but also APIs for handling them under program control. For example, you can wrap an arbitrary function call in condition handlers that will either drop you into a breakloop and enable you to vivisect the program state, or consult the dynamic state and compute which of several restarts to activate in order to transfer control to a path of your choosing.
I'm banging up against HN's length limit, but the above, I hope, goes some way toward answering to your question.
Very nice description of the power of the Common Lisp REPL.
Did you have the pleasure (?) of working on any of the lisp machines?
I got into Common Lisp in the early 90's and even then, with GCL and CMUCL, it blew my mind at how much more productive I was compared to my years of C,C++ just due to the one fact that I was now freed from the edit, compile, run cycle, let alone all the other benefits.
It's truly a joy. I don't have a full grasp of why others don't make the deep dive. I suspect it just had too much a learning curve nowadays, with Emacs being almost a requirement, and developers, like most humans, don't like to slow down for a while while they learn before they speed up.
I feel like I ran out of room before I really covered all that I wanted to cover, but maybe it's better that way. I wrote a tome, as it is. The market for multi-page screeds about how great ancient programming systems were may be limited.
I had a Symbolics Lisp Machine in my office at Apple for several years, and I noodled around with it a lot, but didn't build anything very serious with it. It's certainly relevant to any discussion of the power of full-bore repl-driven programming, though. Now that was a repl!
To be fair, I don't think everyone will find the advantages compelling. As far as I can see, some folks honestly prefer the programming-as-carpentry approach, and a skillful software carpenter can be extremely productive, too.
On the other hand, for someone who is, like me, more effective using the teaching approach, the difference is night and day. I have actually been one of those mythical 10X programmers before, as measured by version-control-system statistics, but only when I was working with a programming-as-teaching toolchain.
You touched upon a point that I think does work against Common Lisp nowadays: it's not super easy for someone to get started with it. I was lucky in the 1980s in that there was a very good Common Lisp on the Mac that was also really really easy to get started with.
Installing Coral Common Lisp was as simple as dragging a folder icon in the Finder. Starting up a usable IDE? Just double-click the app icon. Create a fully-working application window that handled standard events properly?
(make-instance 'window)
Build a GUI? You could either make-instance a bunch of objects and stick them into the window, or you could open a UI-builder window and drag stuff together. Tell the objects what events to handle and what functions to call, and you had a working app.
Deliver the app to production? Choose "save application" from the File menu.
The core of that same Lisp still exists in the form of Clozure Common Lisp, and it's very good, but it's not the same. For one thing, the IP agreement that open-sourced the compiler and runtime did not include the Mac integration. For another, there are a lot more hoops to jump through in UNIX-based macOS than in the old Mac system to achieve the kind of integration that Coral Lisp had. I'm not saying you couldn't do it; I'm just saying it would be a lot of work.
Actually, I think you could make a pretty good try at a Coral-like Lisp, and I think it would make things a lot easier and more attractive for new Lispers. It might even grow the pool of working Lispers, and improve awareness of programming-as-teaching. It would just be a lot of work, and it's not blindingly obvious that the result would pay for itself.
Probably the most straightforward route is to choose a system of this type—-either a Lisp or a Smalltalk implementation—-and learn how it behaves.
The source code for both Lisp and Smalltalk implementations is readily available, but may not help you all that much until you’re at least generally familiar with what the features are and what they do. Once you’re familiar with their behavior and with reading Lisp or Smalltalk code, then you can start rummaging through the code and asking questions of the implementors. Implementors of Lisps like CCL and sbcl, and of Smalltalks like Squeak and Pharo are pretty accessible and generally pretty friendly, and they certainly know how the systems work.
I can’t really speak to Python/Ruby REPLs as I haven’t used them, and I only have experience with Clojure when it comes to Lisps (so I guess I probably shouldn’t even be answering your question due to lack of qualifications), but I believe the big difference is that with a Lisp, the REPL is directly connected to your running codebase.
You can edit a function in a source file and send it to the REPL to be evaluated (or you can type it directly into the REPL if you really wanted to). Once the code is evaluated, it becomes part of the running application.
If you want to test a function that connects to a database for instance, you can invoke that function from the REPL connected to you running application and it will talk to the database.
As an extreme example, you can connect a REPL to a running Clojure application on a remote server and edit the code as its running (!). I like to joke that this obviates the need for Green/Blue deployments in production. ;-)
A thing I find myself doing a lot in Ruby is redefining a method (monkeypatching) to see if it fixes a problem:
class Foo::Bar; def something() something_else; end end
Working in Clojure is like if the whole environment were built with the expectation that you would be doing that all the time. Instead of typing into IRB, you just send it to the REPL from inside your editor with a single keystroke.
The REPL has spoilt me. Working with languages without a powerful REPL, even ones with a shell like JS/Python/Lua/Ruby feels like a regression comparable to moving from zsh to windows cmd.
I have managed to program a REPL in Lua for game development. I also added a feature that lets you reload any single Lua module at runtime, thanks to late binding through tables. Nobody seems to standardize on a good way of doing it so I had to roll my own but I've saved countless hours of restart time doing so and had fun programming at times.
There really, really should be a standard protocol for communicating with a REPL in a running program like there is for IDEs in the Language Server Protocol. Maybe that way the concept would become more mainstream if you could just take any REPL client and walk up to a program with a REPL server and interactively explore its state. The closest thing to said protocol is nREPL which is what Clojure uses, but there are a lot of nonstandard extensions to the protocol for edge case features only applicable to Clojure, so almost every client assumes you're using Clojure and it greatly reduces the viability of using it for Lua or other languages. It doesn't have the backing of something like Microsoft either, I guess because this REPL thing lives in its own world with the Clojure userbase.
I opened an issue on Clojure's REPL integration with Emacs for supporting more languages[1]. I wanted to take advantage of the effort spent on Clojure's client side tooling for things like the command prompt and state visualiser but apply them to at least Lua and allow others to add their own support for other languages. It would be a really big effort but I've already seen the value of it by integrating a REPL with projects written in Lua. As you said I can't ever imagine using a dynamic language without one again.
It is, it's just part of a project I'm working on that uses Love2D and integrated with it a bit tightly. But I think it could be useful.
Actually when I think about it the thing I made probably isn't a "REPL" in the Clojure sense since all it has is a command line and module hotloading; there is no namespace support and the command line evaluation does not happen in the context of the program, but you can basically call a function that gives you the entire global state of the game to mess with and see changes instantly. Maybe that's a better approach than having global state shared between both as Lua has no namespace support built-in, and I don't want to hack it on to keep things simple.
Here is the server which listens for commands using a custom protocol. I want to use nREPL at some point but that requires merging some changes into external projects with the client side tooling.
Here is the code which handles hotloading. It's complicated but I had to write the system so that it also handles sandboxing external mods and hooks into Lua's module management, running special checks to see if the loaded code is not part of the base game. You can probably get away with the version in lume if you don't need as much. But the point is that this is what ultimately allows changing files and seeing the changes instantly without restarting the program.
If you think zsh to cmd is a regression you should try out bash. It's vastly superior to zsh in literally every regard. If you still aren't stratified use fish.
Genuinely never got how people use zsh. It's the slowest and clunkiest shell I've ever witnessed it's fucking slower than most electron gui software.
Personally I've found zsh's features to massively outweigh it's "slowness" (although I haven't really noticed zsh as "slow", per se)
`esc, esc` to prefix the line with `sudo`, better tab completion (both the navigatable UI with arrow keys, along with the intelligent tab completion (I type in "./llo" and it finds the file "./helloworld.sh")), combined with the massive amounts of customization and modules you can install makes it a shell killer. The only shell I've found that I like more is Windows' Powershell, however that's mostly because I __love__ the Powershell language and syntax.
However I will admit that almost all of what I just complimented is part of oh-my-zsh, which is a community add-on for zsh which makes the experience so much nicer. In my time I've used bash, csh, pwsh and zsh and if I could, I'd always use zsh for termanal interactivity and pwsh (Powershell on Linux) for scripts.
Hmm. Curious. How is it slow for you? I switched about a year ago to see what all the fuss was about and piled on as many addons as I could and yes it was slow - to start, but not to use. Which, if you like to pop up new terminals all the time, defintely is annoying. I then pruned it down to what I actually used and got the startup to less than a second, which is just fine for me as the benefits greatly outweight the little blip. No noticable slowness, except for that initial startup. And this is all on a circa 2008 Thinkpad clocking in at 1.8Ghz.
A shell is a kind of REPL, specifically built around accessing OS level commands and features. Language REPLs offer that same kind of access but for your language and language runtime.
I started learning Clojure yesterday,I've gone through the docs here and have a grasp of the language. Does anyone have some good resources for further learning? Maybe some high quality tutorials? I'm mostly interesting in web / apis
Eric Norman at PurelyFunctional.tv has the best video courses on Clojure that I've seen. I found it super helpful to pick up techniques on how to get things done. Some of clojure's concepts feel debilitating at first (immutable data structures and pure functions), so watching a seasoned programmer/trainer at the REPL building is necessary. He also goes through setting up Emacs and other environments, along with using Reagent (React for Clojurescript), etc.
I don't think there's a single best Clojure book, so I'll recommend several (all available on Safari if you have that). For general language, syntax, idioms:
* The Joy of Clojure (whirlwind tour of the language and Clojure philosophy.)
* Brave Clojure (A very wordy, hand-holdy approach. I really like this one for dense topics like core.async which can take time to wrap your head around)
* Programming Clojure - just an all around solid reference. Once you've got a command of the language, this becomes more valuable imo .
Now, there's a large gap between knowing Clojure's syntax and writing effective programs in the language. Clojure is not Python, or Haskell, or Java, and, as such, you don't build programs in the same way. So, as a final recommendation, I'd suggest the book Clojure Applied. This book deals with going from "I know the language" to "I know how to build things in the language"
For the web, get ready to beat your head against a wall for hours and hours on end ^_^. Lisp's "small pieces" philosophy is both a blessing and a curse. Outside of a framework called Luminus, and a few leiningen templates, the way to do things in Clojure land is by stitching a bunch of ad-hoc libraries together by hand, which means a lot of evaluation of not-too-popular github repos.
Once you've got everything dialed in though, it's an absolute joy to use.
For the newbie Clojurist trying to do web stuff I'd highly recommend Luminus. I don't agree w/every choice the framework makes, but those are minor details compared to the choice paralysis one tends to suffer without some experience.
Clojure is too good to miss out on because of stuff like that.
If not already mentioned, John Stevenson's Practicalli site and YouTube channel are very good and active. If you are going down the Emacs route for the first time, his contributions in and around and explaining Spacemacs are pure gold
Living Clojure is my favourite learning book on Clojure
Also I recommend making friends with someone who can show you their structural editing and REPL setup, preferably more than one friend because it's like trying on clothes
Programming at the Haskell REPL is a great experience, whether it's if you're exploring data, testing out a parser, or running an effectful computation with a given state. On the other hand when a language is impure like Scheme or Clojure I've often found myself needing to reload the REPL again and again to get to a clean state.
I think obviously it's much _easier_ to create a mess in Clojure this way, but I think in most sane codebases the pure and impure stuff is quite neatly demarcated. Generally people have learned to structure systems as some sort of root data structure containing various components, which can be started, stopped or reset at will. You have to put that in place yourself, but I've not worked on a project where it's been an issue in many years.
For what it's worth, I have found dynamically loading new libraries at runtime easier in Clojure than Haskell, which is another source of REPL friction for me, although I rarely use the latter these days.
Here is a question that I have about Clojure and its REPL. I have tried 3 times over a period of a year or so to get it to behave like a common lisp REPL and every single time I have hit a brick wall called project.clj and/or deps.edn (which open a terrifying black hole to maven). My simplest use case is to be able to call `(expt 2 10)` from a bare clojure install without leaving the repl. As far as I can tell it is impossible to achieve this. I finally gave up and wrote a gentoo ebuild to install the numeric tower just to see if it was possible [0]. What I learned was that I have no idea how clojure libraries interact with the JVM. `pomegranate` gets the closest but I'm still fairly certain that you can't install that from a repl without resorting to hackery [1]. I think I understand the tradeoff toward using project.clj and deps.edn to create sane and static environments for a project, but the friction is larger than nearly any other language that I have tried because you are practically forced to create a project structure in order to do anything.
I agree that historically it’s been a bit complicated. On OSX you should be able to install using `brew install clojure/tools/clojure, the typing `clj` on the command line
Or just use repl.it
https://clojure.org/guides/getting_started
Just install lein and do lein help new to see available templates. Or lein new myproject to get the framework for your own project. Then Cd into that folder and do lein run
I'm a big fan of REPL programming and miss it a lot when I work with compiled languages. There is definitely a 'right' and 'wrong' way to do it though. REPL programming doesn't replace the need to write tests and it actually allow you to write those tests faster as well as build more realistic faked versions of upstream or downstream dependencies.
Programming in the REPL is something I do all the time and generally enjoy, but I sometimes feel like the actual design of the program then becomes too focused around REPL usage. Too much abstraction can harm performance and waste time. The REPL encourages this by making you think about the internals of your program as an interface that will actually get used, when in reality some function may get called twice ever from within your own code, and isn't really meant to be exposed. But because you are repeatedly invoking it manually via the REPL, it feels like an interface worthy of considerable design.
Maybe it is, and maybe it isn't. I've just noticed this tendency in myself when working in REPLs. Not every part of your program is an API.
I've enjoyed noodling around w/ Javascript in various simple REPLs. I love the feedback, but it also feels a lot like using "ed" versus a visual editor. I find myself writing code in a text editor, then pasting bits and pieces into the REPL.
I'd love to have tooling that let me develop in a REPL, but also had a text editor to let me edit objects / functions / etc in that manner, too.
Well, with proper REPL integration, you never really type/copy/paste into the REPL.
You write your code in a source file like normal, highlight what you want evaluated, and have the REPL evaluate it.
Clojure has a great REPL because you are effectively working inside your own codebase, as it runs. Its a strange idea to wrap your head around, but very cool.
You can get something similar to this approach in JavaScript using Quokka[0]. It's not as complete as a Clojure REPL, but its still quite nice.
If anyone hasn't tried Clojurescript's browser REPL, it's super easy to use now when using Shadow-CLJS for your project.
When you boot shadow up it'll create an nrepl server for you, you can just connect to it and invoke `(shadow/nrepl-select :app)` and it'll latch on to the running application `localhost:3000`. And it works really well, very awesome.
The clojure REPL is nice, don't get me wrong. But I think the reason people rely on it more than in other languages is because of clojure's slow startup time. If you could just make a change and run tests immediately, you'd see much less REPL abuse.
Common Lisp is not terribly slow to start, and people rely on its REPL when programming in it. Even Python's REPL is quite popular, and it's not slow to start either.
The REPL is relied on because it's useful for exploring parts of a system or program. It's like being able to both dissect and vivisect the system without having to go through all the normal issues of setting up a debugger and instrumentation. The environment and program is just there.
I never really did much REPL programming until I started using pdbpp with a bunch of aliases and customizations in pdbrc and pdbrc.py (the former does aliasing[0] the latter does setup before boot) it makes iterating with a test suite so much faster.
[0] For example:
alias cc import pdb; pdb.disable()
alias ppr pp response.json()
I'm not sure this is correct. The REPL 'phase' of feature development tends to be useful before you want to ossify things in tests; it's excellent for exploratory thinking, and it encourages you to build a small suite of tools and state to tinker with, whereas tests cut through to use the minimal required tools and state to formulate the completed, trimmed down thought.
The repl is another tool. I can and do run my tests in the repl. I can set my tests to run everything I change a fill, now the repl has enhanced my tests.
The reason the repl is widely used in clojure is because it's the fastest way to get feedback on ridge code.
I draw a sightly superficial line between ridge and dynamic code. Ridge code would be like an API call to get very specific specific results, here the repl is better then a unit test, for it would be too much for code and time to capture your specific API call results, namely because you haven't really tested anything much larger than a couple data points.
So, it's the opposite, unit tests are often used because languages lack a repl.
I run with lein test-refresh on my clojure code, as soon as I change code in either my src folder or my test folder, the tests are run.
It's instant. There's no waiting on the startup time here at all, sometimes its so fast I don't believe the tests have actually run (so I break something and change it back to confirm).
It's a very very fast way of developing.
You can see me breaking a test here and fixing it:
Once you get over the learning curve, you really begin to see the productivity gains in your day by day to the point that any language without those capabilities fall short. Combine that with a world where most jobs seem to place a higher priority on other things than programmer productivity such as programmer fungibility, and your left with very few choices of places to work that allow you to be at your most productive.
Say only Common Lisp had floating point arithmetic (or first class functions, or...) , and most other languages only had integers. You'd probably feel a bit hamstrun when tasked to do certain tasks in those languages.
Not the GP, but for me it's that Common Lisp's REPL (I have very little experience with Smalltalk) lets me construct programs in an incremental way to a degree I can't easily achieve with most other languages (especially batch compiled languages).
Suppose you want a program (whole program, can't pipe pieces together via your unix shell) that downloads a file from a URL, parses it, does some computation, and prints a result.
In CL, I'd first use the REPL to download the file (so it's resident in memory). Maybe I extract that download code into a function at this point, it's not that complicated. Then I'd start extracting bits of the data piece-by-piece on the REPL. If it's JSON, there's a library for it, but what if it's something that no one else has published a library for yet? It's in my head, so I'll go with it: I had to write a parser for a bespoke data recording format. Everything was in 7200 word blocks (16-bit words), and the first one was a header which said how many other blocks to expect and some other things. So I have my data, still in the REPL I can use `subseq` to grab that header. Then I can break out all the data into it one piece at a time (in this case it was all in fixed positions, except some parts were strings that may only use a portion of their range). So I have written a dozen lines of code that look something like:
> (parse-integer (subseq data 75 79) :radix 16)
Turn that into a function in my source file, make a class or struct that can hold all of this data in a sensible way, repeat for each function (extending the class as I go along if I didn't enter all the fields the first time). I can then test the individual functions and make an instance of the record-header class:
> (make-instance 'recording ;; all those lines)
;; it worked!
Take that and make it a set of functions in my source file like:
I can then repeat this process for how I develop the code to handle each of the data blocks. I don't have to contort myself by writing multiple compilable source files to test each part. I don't have to create a bunch of flags to my program to run subsets of it (because I haven't really finished the whole thing yet, or want to test only a specific part of the parser). If you have a good test framework (I've been pretty happy with C# here), you can approach this idea more easily. But it's really nice to have both: A REPL and a solid test framework.
I don't have to think about writing a function/method for everything, many times I really just want to try out a single line of code to see what the result is (especially when exploring a new library, or prototyping). A good REPL (like CL's) lets you write single expressions and see what happens, and then you can convert that into your real program later. And these single expression executions can become my test cases if I'm doing it properly.
All this said: I generally use org-mode in emacs and source blocks (commented about that elsewhere in this discussion). But I use it in the same way I described above, the main difference is that everything is already located in my source code (but not all will be in the exported .lisp file) so I have a solid record of my experiments. Much like using Maple, Mathematica, or Jupyter notebooks.
This is a PEBKAC, not a problem with REPL-driven development. He had all the test bodies written as input in the REPL, he had all the test results printed as output in the REPL. Why didn't he turn these into unit tests as he programmed?
That's a simple lack of foresight on the programmer's side that is then blamed on the tools that were used.
I knew his article would get misconstrued this way to clarify he almost certainly will be using a REPL in Clojure (even running tests happens through a REPL in Clojure) but I think what he's trying to convey there is that he's not designing in the REPL but rather with TTD
Which honestly is fine but most people reading the article won't be in a REPL based language like smalltalk or lisp so will just understand REPL bad TTD good
Not that I'm an expert but I guess he was doing it the wrong way. I just use (comment...) blocks to preserve what I entered into the repl. These blocks are also easily turned into test cases. It feels kinda similar to tdd.
They purposefully called a method which did NOT exist, used the debugger to partially unwind the #doesNotUnderstand stack, implemented the method, which then returned the expected result as if the method had existed all along. Benefits of super late binding I guess.