1) Most popular frameworks you would encounter run on higher level dynamic languages such as PHP, Ruby or Python. These languages are a decent bit slower than performance optimized languages. The languages towards the top are all C, Java, Go, etc.
2) The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.
3) In general, it's more likely you will have heard of older frameworks than newer ones. The frameworks towards the top are generally all less than 2 years old (excluding Java servlets which have just always been decent). This means they make use of newer designs such as non-locking and asynchronous connection handling.
When does speed become relevant? It's actually always relevant as soon as your product gets traction. For example, 1 server at the top of that chart may be able to handle the same load as 10 servers towards the bottom. If you're using a framework at the bottom and paying heavily for 10 servers, you could increase your startups runway substantially by swapping. However, swapping may cost more in engineer-hours than the servers.
Basically, the relevance of speed is directly related to your margin. If your margin is high (investment banking maybe?), then speed is almost irrelevant, you can afford a whole building of servers. If you're competing heavily on mass market web apps, speed is fairly critical as you can out price competition.
To add to this, I'd encourage you to look at the source code used to implement the tests in the higher performance frameworks. Ryan's third point is very relevant: you may find that modern frameworks leverage both high-performance platforms and much of the pragmatic thinking that was introduced a few years back in the frameworks with which you are more familiar.
I would go even further - performance is relevant even if your margin is high, because high baseline performance means you can trade off performance for speed of development and/or business considerations elsewhere. Scaling also doesn't come for free and it's harder, not merely more costly, to manage more servers.
But for me all this seems quite biased towards small Frameworks/Platforms. I will hardly ever serve one JSON response form full stack framework. If I just made some C server that would do those tasks very well it would be first in this list but very far of anything of value.
Albert Einstein once wrote, “Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.”
> But for me all this seems quite biased towards small Frameworks/Platforms.
If one framework does less than another while still fulfilling the requirements of the test, then it ought to outperform that other framework, right? I don't see that as a bias; it's the expected result.
I said "does less" rather than "is smaller" there. I think that's an important distinction. It's theoretically possible for a framework to provide a feature at zero cost to users who don't use that feature (in responding to a given request). Some frameworks don't do that.
> Basically, the relevance of speed is directly related to your margin.
True though the average PHP developper is usually cheaper than the Go or Java one.
Paying people is usually more expensive than buying server instances on Amazon.
High end PHP dev is as hard as Java .. The problem is that the high end PHP frameworks are at the bottom of the list...
> The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.
Not really Spring,Struts,etc ... are not really low level.
There is a paradigm shift today. There is more and more demand for realtime apps. And these cannot take advantage of http proxying or caching. So the solutions that perform badely at concurency will not be seen as relevant in the future.
