I think the reason most of us dislike the ecosystem isn't because therre are many libraries and tools in the stack. In the C-world, every time you type 'make' I am probably executing two dozen different programs, so we are very comfortable with "Make each program do one thing well".
The problem that most of us dislike the javascript/web ecosystem is that those set of libraries and tools keep changing every couple years. As someone who doesn't do this as a full time job but once in a while has to poke his head out to the other side, it's really hard to keep up with even the acronyms. For example, I know that we are using npm, but now you say we should use yarn?!
I don't buy that either. The youngest thing on that list is yarn, but aside from that everything is a few years old at least.
And nobody is saying you need to use anything. Npm works fine, bower still works fine, hell pulling down tarballs still works fine. You don't need to use the newest thing if you don't have the problems it's trying to solve.
Yeah, from the outside I can understand the "everything keeps changing!!" complaint, but having been involved in JS for a few years, every single tool that got replaced was replaced for completely valid reasons, that couldn't have been solved incrementally by improving what was already there.
Because that's what leads to monumentally difficult to use tools.
Instead of choosing between gulp or grunt, you'd need to choose which config flags you want for this hypothetical "gunt". So instead of having simple to use tools (but a bunch of them), you have a few tools which take weeks of learning before you can even scratch the surface on them. And it leads to tools that have settings that nobody knows about because nobody uses them, and weird security holes and issues because of legacy support for an option that was added 3 years ago which when combined with a new flag causes everything to break...
Gulp is very simple, it's like 4 functions, and like 3 or 4 "conventions" you need to learn. There is no config, there are no settings, and there isn't any baggage. And that's the theme with JS tools. Rather than bloating current tools, you make a new one. Switching between them is easy, as they are all very trivial in nature. The power comes from combining those trivial tools.
Even if that were the case I would think learning powerful tools over the span of a few weeks would be worth the effort if you had the reliability that these tools would exist for years to come.
It seems like the rapid change and throw-away-ability of the tool ecosystem reflects the final product as well -- and maybe that's the large difference which is emergent from web technology? What is the average lifespan of a web application?
I don't think the past few years will be representative of web development in the future. The community has had a lot of churn because basic questions (How do I ensure my code has what it needs to run? How do I include dependencies? How do I efficiently deliver my code?) weren't answered, but now those questions have robust, widely used solutions, we don't have everyone figuring that out for themselves (and writing throwaway tools).
I think most of the current crop of tools that are in vogue will stick around for a while. There have been a few challengers to Webpack offering feature improvements (Rollup, Broccoli), but the ideas they introduced were added to Webpack, rather than replacing it.
I use maybe 3 main tools for my build system. Gulp, webpack, and babel. And I only use gulp because it is a bit easier to read than just putting stuff in a makefile (which is replacing one tool with another), or just using bash scripts.
If we were going with the "add options instead of adding tools" way of doing things, I might have one tool, but 100 flags and settings that I would need to manage and change, and you can't ignore the flags and settings just because you want defaults, because the defaults would work in a way that's out of date, or there might no be defaults at all.
Just because there are tools out there, doesn't mean you need to use them all. You can ignore everything you don't need in your project. And that includes ignoring the downsides, bugs, security issues, breaking changes, and baggage that comes along with them.
In half the cases parent presented, there were no "existing tools" to update(manual polyfills -> modernizr/6to5, script tags -> package managers like bower/npm, nothing -> eslint/type checkers, writing ad-hoc scripts for your own os -> grunt/gulp).
The rest of the examples are cases of merging two large ecosystems(web development and nodejs) together - that spawned browserify/webpack for bundling, npm for package management.
Many of these tools did try to update and adopt the new schemes. The grunt developers spent a lot of time optimizing their architecture in order to achieve the same speeds that gulp had. NPM has not stopped evolving - major version 3 tries to address a lot of the problems that yarn tackles. Babel evolves to adopt new javascript features.
I see the churn in development tools as a very natural progression, but that might be because I've been following it for a while and have seen and felt the pain points that each of these tools try to solve. I see a lot of effort within the javascript community to remain "compatible" wherever possible - for example, there are tons of guides how to migrate from grunt to gulp, there are matching tasks to ease your migration, etc. Library developers often release both on npm and bower, and craft elaborate "UMD loaders" for their libraries to allow for commonjs/AMD to exist side by side.
All in all, I find living with "javascript fatigue" to not be very exhausting. In our company, we have a wide variety of projects - some use literally no tooling, some use just bower, some use grunt, some use gulp, and our latest ones use the "modern stack" with webpack, etc. All of these projects are still actively maintained and their respective tooling stacks work and provide the benefit they were added for.
The problems the "newer" tools try to solve may not even be problems your development process has. If you've never really suffered performance problems working with grunt, then perhaps you've never felt an urge to migrate to gulp. Maybe you've never been bitten by npm's non-deterministic installations and poor download performance(I really haven't, so I haven't migrated to yarn, for example).
The magnitude of web development projects has ballooned in the last couple of years, and the tools are racing to handle it. It's not surprising that tools that are fit for a large web application with tons of code and a large team aren't really fit for a few javascript files to decorate a mostly static site built by a single developer. The churn is there, if you need it, but you don't have to use any of the newer libraries or tools.
The thing is, in other languages, frameworks and libraries can evolve without being replaced. It's called version 2. I can't help but think that in the JS world, you are more likely to ditch the old project because JS code is more difficult to read and maintain than languages that have more static typing, a somewhat higher entry bar, and a culture that values stability.
In other words, this is the consequence of optimizing for new entrants and writability versus readability and maintainability. It also becomes much harder to funnel talented people into being contributors to existing projects rather than starting their own.
In other languages there are very large projects that are heavily refactored and improved to support new patterns and use cases with regularity. Just yesterday I was using Antlr 4, which has been around for 25 years. But it's been updated many times. Maven was started in 2002, and maven 3 in 2008. It's stable and production quality. The openssl library goes back to 1998. Vim goes back to 1988 -- also rock solid and reliable. We're using a networking stack that is pretty damn old. But in the JS world, I can't help but think they would never be able produce anything that reliable because 6 years later they would be on their 6th re-write of Vim. And no, it's not (IMO) because other languages have corporate sponsors.
Javascript came out of the webspace which (was) primarily throwaway code. It was a bad thing to have an old website. They prioritized rapid prototyping, which was appropriate for that business case. However as Javascript is maturing and branching out to server-side programming and at the same time the client side becomes more complex, you can't keep re-writing all your tools every couple of years, or even every 5 years. You need tools that last decades, even as they are incrementally improved. If you don't have that, you are going to be subject to increasing exponential costs that are going to weigh very heavily on the language. Soon, you'll need to spend almost all your resources just re-writing stuff. It's no different than the developer that builds non-sustainable housing that has to be torn down every 10 years but is really cheap to build. Long term, a city built that way will necessarily be smaller in size and more expensive to maintain than a city built with housing that is more expensive to build up front but lasts a lot longer. Exponential decay is going to catch up to languages that like to keep re-writing all their libraries.
More importantly, I don't see an awareness on the part of the JS community that this is a serious problem that they need to tackle. They look at all the new libraries and the fact that they have the most code on github as a good thing, instead of as a warning side that they are already spending way too much time re-writing stuff.
So when there is announcement -- here is your standard webstack and it contains a tool less than a year old, a lot of people are gonna say "WTF", and explaining to them that Yarn has some new feature that NPM doesn't kinda misses the point. Of course it has some new features -- but still -- "WTF"?
> However as Javascript is maturing and branching out to server-side programming and at the same time the client side becomes more complex, you can't keep re-writing all your tools every couple of years, or even every 5 years.
The client side becoming more complex is the reason why new tools keep being developed.
> If you don't have that, you are going to be subject to increasing exponential costs that are going to weigh very heavily on the language. Soon, you'll need to spend almost all your resources just re-writing stuff.
No one is forced to rewrite their code when a new tool is released.
> They look at all the new libraries and the fact that they have the most code on github as a good thing, instead of as a warning side that they are already spending way too much time re-writing stuff.
There is a post, somewhere around here, that characterizes the JS culture as being one where shipping is the win condition. And I think this reflects what you describe better than the idea that other cultures merely value stability; JavaScript seems to value Github stars. And, as you say, that leads to a desire to rip out, rewrite, and own things, even if it doesn't necessarily make sense.
Just google "yarn vs npm" and you'll know the answer...
the tools change because the technology is evolving, despite node the front end is still the driver, and mobile phones / browsers / etc are constantly being improved and evolved, and the ecosystem keeps up with it. Would you rather it all stayed static like in the C world and then you program a 2016 smartphone with a 2006 stack?
The answer to your question is yes. I would rather the stack, or at least most of it, stayed stable for 10 years. The problem seems to be that the JS community keeps re-discovering the wheel at every step, instead of actually looking at other ecosystems and learning from them.
How log did it take for a sane build system to show up? Or a sane package manager (with dependency resolution, upgrades, checksums, signature verification, etc.)? Why is Grunt/Gulp a thing when we have thinks like make/CMake?
The technology is not changing, in the sense that the state of the art is not really changing. In the past 15 years the biggest paradigm shift has really been towards single page apps, ES6-7, and TDD. The rest has existed as state of the art in many other ecosystems, but the JS community is notorious for having a bad case of NIH syndrome, so it's taking forever to get to the state of the art.
The good news is that it will slow down at some point. The bad news is that it probably won't be for a few years.
> Why is Grunt/Gulp a thing when we have thinks like make/CMake?
What a ridiculous statement. It's like asking why .NET doesn't use the JVM or why Haskell doesn't use the C compiler. Grunt/Gulp solve web dev specific problems, make is a different kettle of fish.
No, the problem is never improving the existing software but rewriting a new solution from scratch every 3 months [1][2], that, while it solves the shortcomings of the previous tool, now introduces a complete set of new problems. And people that actually have stuff to deliver are always playing catch up and fighting the bugs of alpha quality software.
I feel strongly about this, especially when reading a step-by-step tutorial, that's most probably intended at frontend "newbies", that for example is pushing Yarn: I played with it the other day on a working VueJS application, and got hit by a bug, causing yarn to delete important files in node_modules. My code wouldn't work anymore. Really sorry that I can't find the bug report anymore to prove my point. The workaround was going back to npm.
Teaching people alpha quality software is irresponsible. If you want to teach, or have stuff to deliver, use the battle tested tech. Experiment when you know how the pieces work together and have lots of free time.
2: EDIT: this is a big problem of our industry, it happens in all environments, but the JS community is the worst IMO. Desktop Linux is the second worst offender.
The problem with continuously changing technologies is that it doesn't allow you to become a master of that stack, so your constantly relearning how to accomplish the same thing. It's inefficient. If I build my applications with well established technologies, then I can do it quickly and accurately. Sure, I might miss out on some cutting edge benefits, but my shit works, always.
However, if i switch to JS, I feel like I will have to not only relearn how to do things I already know how to do, but I will be stuck in a cycle of continuing education and won't ever be able to expand my skillset beyond those things.
That's the state of the computing industry. 80% of the continuing education is about learning to do what you already knew how to do in a newer, hotter stack (but not necessarily better). 20% is about learning new paradigms. And I'm being too generous. It's probably about 5%.
Suddenly being a medical doctor sounds much less stressful. I really feel some of us chose the wrong career path. Imagine having high social status, having almost always some meaningful work, all for debugging meat machines that have lots of failure modes, but most of them well understood?
Having experience in both worlds, I can assure you medicine isn't what you seem to think it is. Social status, and income aren't so grand these days, and working conditions are bad for a lot of practitioners. And "debugging meat machines" is anything but well-understood in the majority of serious cases.
I know working with Javascript can be an annoying proposition. I'm maintaining a web app I wrote a couple of years ago in vanilla JS and gotten to be kind of a hairy beast. Probably would be better to rewrite it, but it works and for all the warts at least I know where they are. Maybe I'm just too lazy to take the time and effort to recreate from scratch. I figure by the time I got it working, I'd always be lagging the newer, "better" stuff anyway.
Interestingly, your comments about education have a parallel in CME (continuing medical education). There are real paradigm switches that appear from time to time, but not as much as you'd think. Most changes have to do with small refinements like new medicines that are variations on the current ones, theoretical or research findings, administrative or procedural issues.
So yeah, the really important information approximates the 5% figure you cite, and a significant amount of "updated" info is only marginally different or new, more or less equivalent to the "newer, hotter stack" idea you mention. Sure seems all that greener grass still needs to be mowed about the same way.
The problem that most of us dislike the javascript/web ecosystem is that those set of libraries and tools keep changing every couple years. As someone who doesn't do this as a full time job but once in a while has to poke his head out to the other side, it's really hard to keep up with even the acronyms. For example, I know that we are using npm, but now you say we should use yarn?!