Jonathan Blow criticized the performance and complexity of LSP in his talk at DevGAMM 2019 titled "Preventing the Collapse of Civilization" (timestamp 42m26s; https://youtu.be/pW-SOdj4Kkk?t=2546):
> In the programming language world, there's this thing called Language Server Protocol that is pretty much the worst thing I've ever heard of. There are proponents of this all over, building systems for it right now that are going to be living on your computer tomorrow, or maybe even today. As far as I can tell, it's basically a more complicated, slower way to do libraries.
> Say you've got an editor for some programming language and you want to be able to do stuff that we've been doing for decades already. For example, look up the declaration of an identifier by clicking on it, or have tooltips that say, "What type is this value?" Well, they say the way you should do that is—you know, you have your editor and then it's a hassle to make plugins. This is the made-up problem. It's a hassle to make plugins for all these different things. So in order to standardize, you're going to run a server on your machine. Then your editor talks over a socket to the server, and the server talks back and gives you the answer. This approach has now turned your single program into a distributed system.
> The flaw in this whole line of thinking, that none of these people seem to actually think about, is that there's nothing special about looking up the location of an identifier in your code. That's just an API, like we have all the time for everything. So the obvious next step, if you're saying that we should architect our APIs like this, is to do this for other tasks. Now your editor, or whatever program, is going to be talking to multiple of these things. If you ever want to author anything for this, you now have to author and debug components of a distributed system where state is not located in any central place. We all know how fun that is, right?
> But of course, libraries are not that simple. Libraries use other libraries. So what happens at that point is you're running all these servers on your system, and who knows, some of them are going to go down and have to restart. People are synchronizing with each other—no, this is a disaster. And people are actively building this right now, while we're spending all this time overcomplicating stuff that we used to be able to do in 1960.
I don't care how many credentials Jonathan Blow has, this take is pure bs. "we're spending all this time overcomplicating stuff that we used to be able to do in 1960", yet he takes the time to explain "tricks" on how to rename variables [1], while modern tools (not necessarily LSP) let you fly through the code [2].
The environments we work in shape our biases and what we find good and bad.
Game programmers spend their careers writing C++ where, yes, you really need tricks like this to rename the variable, as the language is so fucking insane that ALL tooling gets it wrong - and the consequences of getting it wrong can be disastrous.
The most widely used options for C++ are VS's IntelliSense and clangd. IntelliSense is utter garbage, I never trust it for renaming things. clangd is better but can still get things wrong if your build process is complicated your compile_commands.json doesn't perfectly reflect it.
Managed languages have fantastic tooling with perfect context of your project, but they're also managed and are not applicable in a lot of environments.
> clangd is better but can still get things wrong if your build process is complicated your compile_commands.json doesn't perfectly reflect it.
Yes, clangd is accurate but when you have different build variants only one is reflected in the current compilation database. clangd will be perfectly accurate for this current variant, but blind to others.
Would clangd support a merge of several compilation DB into one? If a file appears several times with different options (typically include paths and defines), would clangd handle all variants in parallel or just pick one (first or last)? I haven't tried (yet ;).
It's manageable, and sometimes I'm reverting to pure "text level" changes or searches to work around this.
rust-analyzer renames over LSP are perfect the vast majority of the time. The dichotomy among static languages isn't managed vs. unmanaged, it's C++ vs. not-C++.
There's some truth to this! I strongly encourage to architect language servers as libraries first and foremost, with LSP being a relatively thin layer on top of protocol-agnostic library, and only _one_ of the ways to consume the results of the analysis.
At the same time, libraries, sadly, do not actually exist. I can not easily cook up an `.so` and then hook that up with Emacs, Vim, VS Code, and Helix. Moreover, any crash in an `.so` library brings down the whole process, which isn't a good idea for an IDE.
Maybe several years in the future we get an actually good implementation of libraries (I have high hopes for WebAssembly component model), but until then, text-over-stdio can actually be better in practice for some use-cases!
Yes. LSP is the worst thing in the world. But nothing better exists for this use case, so I'm glad that LSP was invented and I can enjoy working C++ completion, go-to symbol and other nice IDE niceties in emacs.
I don't get his point? It basically amounts to "nuh uh, this is just bloat".
>What type is this value?" Well, they say the way you should do that is—you know, you have your editor and then it's a hassle to make plugins. This is the made-up problem
Ok so it's just a made up problem? Like, the "plug in" part or the need for having tooltips/completion etc? If he was referring to the perceived need of a plugin (vs. a more integrated) architecture being just due to a made up problem, the linked article in this thread shows this is absolutely not the case. I'm not sure it's better to reimplement the same features every time for every single editor?
And if he meant that needing IDE features is the made up problem, then I guess... lol
?
>People are synchronizing with each other—no, this is a disaster. And people are actively building this right now, while we're spending all this time overcomplicating stuff that we used to be able to do in 1960.
I get the "old good" aesthetics he always aims for, but come on. Even the most charitable interpretation of this part is ridiculous. Like sure we probably had some IDE features back then, but that's not the point of LSP. The point is standardizing said features. And specifically, the interface to access them without knowing anything about the editor or how the code is written.
I'm sure your proprietary custom made computer was able to get or provide code completion from other ultra closed systems in the 1960s. It would've been helpful for him to be more specific about what exactly they were doing back then. Again, the whole point is to not have to reimplement a sort-of-compiler for every editor and whatever quirks it comes with.
He is not even doing the regular "YAGNI, it's bloat", it's just downright weird and devoid of actual arguments imo. I'm not even saying he needs to propose a better way to do it, because he doesn't even seem to acknowledge the point of language servers
He's saying (without saying it) that LSP is another instance of the "microservices everywhere" hype, and bad for the same reasons -- you have a problem, you try to solve it with a distributed system, now you have two problems and one of them is a set of microproblems.
I don't think he is arguing against a standardized interface to language-specific logic, but arguing for that standard to be a library interface, not a distributed system.
He doesn't say why a library interface would be better. That's why I'm saying he doesn't even address the main point of LSPs, which is that they came in response to have your IDE or editor maybe-kind of plug into some libraries and wish for the best. Just saying that a library interface would be better because it's less complex is meaningless, when we obviously have not managed to get anything close to that for the past 40 years.
It's not super convincing to just dismiss the only solution that seems to have sort of worked by saying that we only had to keep doing it the way that never really worked.
I'm sure the implementation can be a lot better, but he does not provide any argument for why the architecture itself is bad. Sometimes abstractions are actually useful.
I also think it's a bit overblown to call communication with a locally running process a "distributed system". I mean, that's kind of true, but not really any more true than if you were to call having multiple threads in your process a "distributed system".
It introduces _some_ of the failure modes of a distributed system, like unsynchronized / inconsistent data, but not all, like network failures. I think he does have a point there -- as a user, I should not have to care whether an IDE talks to a library via its API or to a language server via LSP, but here I am, watching WebStorm to act up and complain that the TS language server crashed.
One thing that Blow misses IMHO is to what extent libraries introduce these failure modes _too_: A library can crash as well (and take down the IDE with it), and multiple libraries that deal with the same data can get inconsistent. That's actually a problem of quality control, integrating code from multiple third parties, and so on.
Security, too. If I’m editing a .env file with a bunch of passwords in one buffer, or maybe some sensitive data I copied and pasted into an empty text window I’m using for temporary notes, I don’t necessarily want some random code written by some 3rd party running in the same process space. I mean, most of userland is pretty much written by various random people already, but I might trust a language server to tell me that I forgot a semicolon in a shell script without trusting it with the contents of /etc/shadow.
The reason is probably that dependency management gets much easier (or at least can get much easier). E.g. using any module in VSCode that requires native code (sqlite, treesitter etc) will have to work with that specific version of NodeJs for theat specific Electron version. Also quite a few languages already got implementations of parsers in their native language, porting them over would be a big effort.
I agree, and mentioning libraries seems like a non-sequitur to me. It’s handy to be able to integrate a language server written in Lisp with an editor made in Swift, for instance, without coming up with a way to link them as shared libraries. That sounds like a nightmare.
He is trying to say that LSP should be implemented as a set of libraries (like liblsppython.so). Your editor compiles with these libraries and provide API for plugin authors.
It's not unreasonable to expect an editor's developers to know how to link a native library.
That’s the part I vehemently disagree with. There are servers written in a whole lot of languages. I’d bet there are Java servers written in Java. How are you going to link that into an editor written in C? What if the server’s written in Python? A shell script (crazy but legal!)? And even if you solve all of those for C, what if you want to write an editor in Swift. Now you have to implement all those shared library linkers again in that language.
Oooorrrrr, you could communicate with another process via stdin/stdout and call it a day.
> There are servers written in a whole lot of languages.
I believe his opinion is that there shouldn't be LP (since he's against to use servers I remove the S from LSP) written in language that can't be compiled to xxx.so. We're talking about Jonathen Blow after all.
It make sense if you view LP as a part of an editor. But in our reality the language servers are often maintained by that specific language's community, not the editor devs, so it's probably not the best idea to expect them all to be willing to write in low-level language.
Two managed languages will be hard to integrate, but if the library is written in C or in a language with a C ABI, is possible to call it from pretty much any language. The question is whether you would want to run it in the same address space as your editor even if you could. Clang is more stable these days, but I still see clangd crashing from time to time and it would be very annoying if it were to take down my editor every time.
So in the end even if you had the option of running the language plugin within the same address space, probably you wouldn't want to. A better RPC protocol than JSON would be nice, but a standard is better than no standard.
You can implement it ("or something like it") on any system. In fact there is a plenty of those. Remoting isn't impossible either (e.g. DCOM, CORBA, GRPC).
No it is not (well it might be with antiquated CORBA/DCOM stack but these are just examples, something newer like GRPC would be easier, the point is that it's solvable).
Also the original complaint was about explosion of language combinations and I just pointed the solutions to this are well known.
Frankly, if your language can't compile a .so/.dll that can interact with the rest of the system in a sane way, use a better language.
I used to disagree with Blow on this point, as stdin/stdout communication really has the arguable advantage that you can write your Brainfuck LSP in Brainfuck, but I've spent way more time than I find reasonable over the last few years dealing with the consequences of the LSP model - tweaking environment variables, paths, command line arguments for the server, server crashes, mismatches of server version vs the compiler, the client picking the wrong server executable, system-provided language vs downloaded by VS code extension......
I'd rather just drop a dll in Plugins/ and be done with it.
Sounds like a cultist who people listen because of their need to be angry at someone. Even after cleaning, this is a word soup with little substance.
LSPs solves two "made-up" problems. Editor and LSP team can be different and independent. Go teams work on LSP without actively collaborating with VSCode team for feature and release. Second, both can run at different places, it allows for things like devcontainer and Codespace. code is built/run on some remote server while IDE is running on local.
Assuming the MxN claim was accurate, yeah. The author of this post wrote a prior one [1], challenging whether MxN was truly the problem LSP was solving. It's also a good read!
Any multi threaded editor is already a "distributed system" by this argument. Maybe he likes his editor to lock up while loading an autocomplete list, but I am against this.
If you are going to have a background thread that calculates results for autocomplete, then it helps to give it a protocol.
> In the programming language world, there's this thing called Language Server Protocol that is pretty much the worst thing I've ever heard of. There are proponents of this all over, building systems for it right now that are going to be living on your computer tomorrow, or maybe even today. As far as I can tell, it's basically a more complicated, slower way to do libraries.
> Say you've got an editor for some programming language and you want to be able to do stuff that we've been doing for decades already. For example, look up the declaration of an identifier by clicking on it, or have tooltips that say, "What type is this value?" Well, they say the way you should do that is—you know, you have your editor and then it's a hassle to make plugins. This is the made-up problem. It's a hassle to make plugins for all these different things. So in order to standardize, you're going to run a server on your machine. Then your editor talks over a socket to the server, and the server talks back and gives you the answer. This approach has now turned your single program into a distributed system.
> The flaw in this whole line of thinking, that none of these people seem to actually think about, is that there's nothing special about looking up the location of an identifier in your code. That's just an API, like we have all the time for everything. So the obvious next step, if you're saying that we should architect our APIs like this, is to do this for other tasks. Now your editor, or whatever program, is going to be talking to multiple of these things. If you ever want to author anything for this, you now have to author and debug components of a distributed system where state is not located in any central place. We all know how fun that is, right?
> But of course, libraries are not that simple. Libraries use other libraries. So what happens at that point is you're running all these servers on your system, and who knows, some of them are going to go down and have to restart. People are synchronizing with each other—no, this is a disaster. And people are actively building this right now, while we're spending all this time overcomplicating stuff that we used to be able to do in 1960.
(Transcript produced by asking GPT-4 to clean up the raw output from youtubetranscript.com: https://chat.openai.com/share/e1bc0e87-f79e-4958-98e7-b93c1b...)