I completely agree with the idea that we are overestimating the impact of tools, specifically, programming languages.
I often see startups that rewrite their software one or more times just because they think that changing the tools is the key to success. They switch from PHP to Python, then from Python to JavaScript then from JavaScript to Go. It's so easy to say "We failed because X language, now we have to use Y".
The case when choosing a given language will give you a considerable advantage over choosing another one is very rare.
The worst thing is that we tend to say that languages suck while completely ignoring all the evidence of the opposite. How many times we have heard that PHP, C++ or JavaScript suck? How many amazing software projects have been built using these technologies?
On the other hand, look at the tools that self proclaim the best ones (lisp, haskell), how many big complex software projects have been built with them?
In the end they're just tools. Yes, they make a difference, but it's tiny compared with all the other pieces of the puzzle of how to build good software.
> The case when choosing a given language will give you a considerable advantage over choosing another one is very rare.
> How many times we have heard that PHP, C++ or JavaScript suck? How many amazing software projects have been built using these technologies?
Languages are better or worse suited depending on the application.
How many of these amazing software projects in C++ are web backends?
Most of entire Web 2.0 boom has been built on higher-level languages/frameworks like Rails/Ruby, Django/Python, and more recently Node/JS.
Its simply not possible for the average software engineering team to build systems as fast and as reliable in something like C or C++.
Similarly, the revolution in applied machine learning we are currently seeing would not be possible without tools like NumPy or scikit-learn and the frameworks that have been built on these.
On the other hand, 3d game engine code is almost universally written in C/C++.
Sure, choosing between Rails or Django will not matter all that much to your stack in the long run.
And its just as important to build a good team around your tech.
But the best team in the world is not going to be able to write a highly optimized game engine in Python or quickly prototype new NNs in Ruby.
New languages become popular because they address some need that was not previously met.
Understanding these roles different languages play is an essential skill in software engineering.
Having (or not having) large successful projects written using some tool is not evidence that it's a good tool. Your implication that it is stands in bizarre contrast to your statement that the impact of tools is overrated.
Force of will, network effects, and other political factors dominate most forms of technical merit when it comes to "success" of a project. This includes such meritorious properties as security as well as understandable code. Let's not give up, OK? Doing a good job is always going to be hard, but it doesn't have to hurt this much, and it doesn't have to be as crappy-by-default as it is now.
As we take responsibility for doing better with our tools, we need to take responsibility for doing better on our tools.
Now while I do believe that some tools are better for some jobs than others, I also think that many times people are just exchanging one set of problems by another without realizing it.
Writing software is hard but to me the hardest part is always taking a random abstract concept from someone's mind (or worse, several people) and converting that into something "real" in a fixed timeline and budget. There will have to be lots of tradeoffs and miscues by definition. We are always making something that doesn't already exist, it is creation and creation is hard.
I do agree that tools (and process and ....) can make it less hard but that requires real, active participation. And I think that is even harder...
I will parrot what a professor told me years ago. The hardest part of writing software is proper requirements gathering. He said it was like pulling teeth. By proper I mean the developer knows exactly what the client wants and the client knows exactly what will be delivered. A "meeting of the minds," I call it. I always tell clients this is the hardest part.
I am not sure I have worked on a project where gathering requirements was a problem. But, most of the time my work has been on systems that have been a work in progress as they were actively being used. The needs are normally abundantly clear. I am not sure I have worked on a product not currently in use.
And also not forget budget. Most think to build like e-commerce like E BAY or next generation Facebook.The mentality yes, but the basic requirement is non. My most problem on development was to many choices and lack of decision making from the client site.
In my opinion the biggest reason why writing software is hard is that the requirements are often not clear from the beginning. Changing requirements can kill a software project. What makes it even harder is when data has to be migrated during the process.
Analogy for clueless manager types: in many cases, software is 99% plumbing. But imagine the cost of changing the plumbing after the house has been built :) Also consider the problem of changing the plumbing while people are actually living in the house.
Writing software is hard yes, but I disagree with the author: I do blame the tools. Every language since C has been created as a reaction. What I mean by this is:
> C++ was created as a reaction to OOP being so difficult to do in C.
> Java as a reaction to memory management being difficult.
> C# as a reaction to Java not being concise enough.
> Javascript as a reaction to HTML not being dynamic enough.
> Go has a reaction to parallelism/concurrency being difficult.
> Rust as a reaction to what C++ has become.
And it goes on. There are many other languages too that you won't hear of that are created as a reaction to the status quo not being good enough either, but those ones I listed were backed by a big enough majority to become known. There is also the Lisp family and the functional paradigm too, which come round again and again in popularity.
All these languages/tools are the problem because no one is sitting back, and pro-actively realising that the entirety of these languages are no good. I am not saying this to start an argument, I have happily been a programmer for 15 years. But I recognise the problems, and they are the same problems, that will repeat over and over again ad-infinitum. I actually started writing down problems I've found and possible solutions and it's already reached over 30 pages.
What it boils down to is that all languages share the same flaw; they specify the what and the how, but never the why. Until we figure out how to encode the why, we will forever be going in circles.
>all languages share the same flaw; they specify the what and the how, but never the _why_. Until we figure out how to encode the _why_, we will forever be going in circles.
I don't believe encoding the "why" is possible.
The mapping from "why"s to "machine code" is not predefined bijective nor injective.[1] It can't be well-defined enough to make a deterministic compiler. If you restrict the set of "whys" to a very narrow set of "understandable" inputs by the compiler, you've basically re-implemented the specifying of the *"whats" again.
Building ever higher abstractions is tractable because it's what we've been doing for decades: combine several lower-level "whats" into higher-level "whats". But encoding the "whys" seems to be unsolvable. Either that or I'm not understanding what you're communicating.
Is this a "flaw"? It seems to me that it's just a normal incremental process.
Wasn't C a reaction to assembly languages not being portable enough? Weren't assembly languages a reaction to machine code not being easy enough to work with? Wasn't machine code a reaction to punch-card systems being too inflexible? Etc. etc.
I disagree, the tooling around most ecosystems is fine you just need someone who knows what they are doing to help guide people in the right direction. Technical leadership is sorely lacking in our industry, and when you are solving business problems management cares about solving customer's issues on time rather that craftsmanship. Therefore people go down the path of doing just enough to get it done rather than doing it right. If you never are forced to do something correct, you are likely never going to be able to build great products. This leads to patchwork engineering and the veritable big ball of mud that most products look like behind the scenes. It's up to us as professionals to simplify, abstract, focus, and enforce standards so that you can build maintainable products faster. Very few problems that we are working on are "difficult" from the technical side of things. Identify when you've made a mistake, think about about how it could be made better, and then apply the knowledge you've gained when building something new. If you repeat that process you will be a lot better than the majority of your peers.
> Very few problems that we are working on are "difficult" from the technical side of things.
I'd say these problems are technically difficult, just not deep. To clarify the difference:
(0) A difficult problem requires a lot of effort to solve. Example: Finding and fixing use-after-free errors in a buggy C program.
(1) A deep problem requires creativity and insight to solve. Example: Inventing and formalizing Rust's borrow checker rules.
Programmers are naturally attracted to deep problems, but not difficult ones. In fact, reducing difficult problems to deep ones is often considered progress, for good reason. After you come up with the right insight, solving a deep problem can be very easy. But solving a difficult problem will always be hard.
Could you elaborate on what you think a better direction for creating a programming language would be? I'm not sure I follow how that would be done in a way that isn't like the list you described. I don't see someone wandering out of a cave and unveiling a p-lang that didn't build off of known drawbacks of past languages.
I think that's part of the problem - none of us have yet the conceptual ability to see what the answer is. We're just scratching symbols on cave walls. Until some massive unexpected breakthrough or insight, it seems like programming languages are doomed to continue circling the same old ground.
I don't know if tooling matters all that much when you think about larger projects like CRMs or kernels. Those are the kinds of projects that can fail just on the initial discovery of requirements since communication between different stakeholders can lapse easily. Tools can be replaced but the stakeholders are there for the lifetime of the product.
It doesn't matter which way you look at it, Java today, after all the work put into catching up, still pales in comparison to the C# of 2007.
Really, C# is a public service to correct for something that for so many reasons should be awesome having turned out so-so at best.
(You may say writing a million getters and setters, declaring types twice, stuffing your code in deep directory hierarchies, and dealing with bizarre licensing/version issues feels normal; that's the Java talking.)
I'm surprised at how little DHH says here. He could have stopped writing after the fifth paragraph. Or just added the phrase "Don't blame your tools" to the headline. Like bad code, this post is verbose and most of its verbiage redundant. Good writing, like good code, is a long apprenticeship.
If your horse was tied to an insanely successful framework? I'm not sure if you are criticizing "horse-tying", or mocking the post itself. In the latter case, I think hacker-news aficionados and the like will be deeply disappointed where hot framework X is in a few years...or, you could do a lot worse than Rails.
I often see startups that rewrite their software one or more times just because they think that changing the tools is the key to success. They switch from PHP to Python, then from Python to JavaScript then from JavaScript to Go. It's so easy to say "We failed because X language, now we have to use Y".
The case when choosing a given language will give you a considerable advantage over choosing another one is very rare.
The worst thing is that we tend to say that languages suck while completely ignoring all the evidence of the opposite. How many times we have heard that PHP, C++ or JavaScript suck? How many amazing software projects have been built using these technologies?
On the other hand, look at the tools that self proclaim the best ones (lisp, haskell), how many big complex software projects have been built with them?
In the end they're just tools. Yes, they make a difference, but it's tiny compared with all the other pieces of the puzzle of how to build good software.