I don't know if I completely agree with this. To write truly performance-critical Go, you end up throwing away many of the language's qualities. Channels are slow, defers add overhead, interfaces add overhead (e.g. I2T), etc.
Don't get me wrong, I like Go, but in my (and others I work with) experience, it's not the right choice for performance-critical systems.
I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com). CloudFlare has it sitting in the middle of every request for some sites (in the form of Railgun, their delta compression for pages). I'm not exactly a perf ninja and got some Go code packing Wikipedia history dumps at >100 MB/s. Dropbox uses Go for, their words, "our performance-critical backends" (https://blogs.dropbox.com/tech/2014/07/open-sourcing-our-go-...). So folks manage to heavy lifting with it.
I see "I couldn't possibly use X for performance reasons" here a lot, as an almost immediate response about a wide range of different tools, and two things come up in my mind: 1) you can always set the standard arbitrarily high. If you're doing AAA shoot-'em-up games or something, yes, use something else. 2) you're often not as good at tools that're new or different. It's possible you could go further with just a few more tricks about profiling or using free pools or whatever, or a little more info about how your code executes or what the runtime does. FWIW, if you hit a specific wall that's a problem for your app, folks on golang-nuts (or StackOverflow, where I've hung out sometimes) are often happy to try and help.
Hope this is more helpful than fussy. Mostly just don't want folks to be discouraged into thinking certain things are impossible when in some cases they're really being done in production out there.
No one doubts that Go is production ready at this point. But the performance requirements of a systems programming language are just very different from the requirements for a web server or applications language.
From my point of view, and the industry I'm in, the latency requirements of a web server stack look completely laughable. That's not to say there aren't challenges in a web stack, because many web servers have to handle orders of magnitude more I/O than we do, and efficient load balancing over a distributed system is extremely difficult and I am fortunate to rarely have to think about that kind of thing. But a requirement like "99% of requests need to have a response time of 200ms or lower" looks like fiddlesticks compared to a requirement like "a round-trip-time of more than 100us will be a large problem and get noticed".
My point is that Google's, CloudFlare's, and Dropbox's "performance-critical backends" don't require the same things as some other workloads. As a more mundane example, the latency requirements of Google's search engine are probably less stringent than the latency requirements of your text editor.
None of this should be taken to mean Go is slow or can't handle 99% of the world's performance requirements. But you can't just say "Go is high performance and fits in all of these company's critical paths. Try harder and it will work." Sometimes it might just not be cut out for it.
>I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com).
Not as impressive as it sounds. Most of it is cached in memory anyway...
Agreed with setting the standard arbitrarily high. It depends on what problem you're trying to solve—just like deciding when to use any other tool. Context matters.
Where I work, we've been trying to build some super-fast systems and things like the GC and even calling interface methods matter. We open sourced some data structures that show just how optimized we're trying to get (https://github.com/Workiva/go-datastructures), but in hindsight, Go might not have been the right tool for the problem.
I disagree with "channels are slow". They may be faster than you think. Defers needn't add as much overhead as they do today; this can be fixed.
I think it really depends on your definition of "performance-critical". I agree Go isn't suitable for all performance-critical tasks, but it covers a vast swathe of them quite comfortably.
I believe this and am counting on it. Go2.0 and beyond should be a solid choice.
You should also note that it is entirely understood that more mature tech e.g. JVM have had the benefit of multibillion Dollar investment by SUN, IBM, Oracle, etc.
I feel it is regrettable that (imo valid and reasonable) criticism of what is currently not up to par with this tech always seemingly requires a disclaimer that "I love Go". I have been using this language since the day it was released. I know it fairly well. I like it. But excessive hype and sensitivity around it is frankly somewhat irritating.
Sommers is an absolute beast, does totally astounding work and is an asset to the community. But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire (https://twitter.com/kellabyte/status/547063455048404992). In other words, pushing the hardware absolutely to its limits is another game (Sommers' game :) ) from simply getting a tricky part of your app performing at the level you need. And, confusingly, "high-performance" or "performance-critical" are terms you might use to describe either task.
For context, the person whose presentation triggered Kelly's comment, MIT scholar Neha Narula, was experimenting with an 80 core machine, and she replied "it's kind of amazing I could push Go that far," and "to be fair they weren't really optimizing for my use case :)" -- her whole presentation is on YouTube at https://www.youtube.com/watch?v=Mbg1COjhsJU (some wild stuff--she got improvements for >48-core machines pushed into the Go GC) and those replies I quoted are at https://twitter.com/neha/status/564569903219634176 .
But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire
We are not all writing web apps. Some of use are in machine learning, NLP, signal processing, etc. where squeezing out as much performance as possible does matter.
In those fields Go is still weak. No autovectorization, no OpenMP, no direct CUDA integration, GC overhead, etc. Luckily, this can often be worked around since cgo is so good. One can write performance-intensive parts in C or C++, compile with the latest gcc or clang and link it with the Go code to drive it. This is often an understated advantage of Go compared to Java, where JNI calls are expensive. But the Go camp always advocate for porting everything to Go (because fast compile times).
Indeed, we agree that Go is suitable for some things people call high-performance but not others. I do not advocate porting everything to Go. Not only is it not always a good fit, but there's a very high bar to clear to justify revamping any perfectly good codebase. (I use Python at work and wouldn't advocate for porting anything.)
OpenMP is for parallelisation, not concurrency. OpenMP is designed for embarrassingly parallel loops that you typically encounter in numeric code, where parallelization is as simple as tagging a for loop with a pragma and (if necessary) marking variables that should be synchronized and/or critical sections.
E.g. you can make the libsvm library parellalized and scale up to many cores by adding two pragma statements.
I have a Go package that attempts to bring some of this functionality to Go [1]. But it's definitely not the same as having OpenMP.
Don't get me wrong, I like Go, but in my (and others I work with) experience, it's not the right choice for performance-critical systems.