Sometimes I really wonder if the choice of the language is just a way to filter out 'boring' engineers. It doesn't seem like they particularly needed to use Go, but look at the attention it gets them for being 'cutting edge'.
(Disclaimer: I'm a former engineer at NYT and was a Go proponent there.)
The reality of the situation at NYT is that every dollar counts. Go is an effective language that enables a programmer to do both "scripting" and "real" tasks without switching between different environments. The email system described sends more email than many companies that only send email send, and have many more employees working on said system. It is one of the few systems at NYT that pays for itself with zero doubt.
The speaker did not know Go before being hired by NYT, and famously solved a Java programming task in his interview without an IDE and barely a typo (the only time I'd ever seen it done successfully, some people can't even run javac or vim).
It certainly doesn't hurt in hiring to say that you get to program Go, but filtering out "boring" engineers is not a chase of "coolness". It is seeking out engineers who want to take a risk and want to be productive.
It's a question that can be done in any language and isn't especially difficult in a shell or scripting language. Every attempt I'd seen (except that one) in Java or C++ was a failure given the time limitation.
It is more of a test of "can you pick the right tool for the job, given limited time and resources?" In this case, a machine gun was chosen to squash an ant. A risky move, but if you can do it, then great.
No, I get that you want to filter out boring engineers. It's just unfortunate that it seems the only way our industry can do this is by changing up the tools and seeing who makes it through.
Some of the smartest engineers I know are not interested in go because they think it solved the wrong problems and prefer rust for something new.
I've been writing in Go at work nearly every day for the past year. I think that of the problems both Go and Rust attempt to solve, Rust solves them better in a technical and academic sense. As someone who prefers C and C++ over Python and Java, I personally would rather use Rust, but I have to say that Go strikes a great balance for general-purpose use. It is as fast to develop and as flexible and extendable as Python. Similar performance to Java but with much less code, better interoperability, and a more robust standard library. More features than C without the needless complexity of C++. Entirely too opinionated and the build system and dependency management are 80% solutions / hacks, but for the most part you can work around them. Most importantly, I think Go can be easily picked up and used by C, C++, Java, PHP, JavaScript, Python, C#, Objective-C, XYZ, ETC programmers alike with very little learning curve (some more than others perhaps).
I think with Rust, you are going to have a hard time training someone who has depended on GC for so long that they have to be explicit about the ownership and transfer of ownership of memory and that it's elegant and cool and performant to do so. You are going to have trouble explaining traits and composability and the syntax around it to someone who is used to using more verbose syntax to implement complicated inheritance trees. Pattern matching and enums are fantastic, but also foreign to most of the programming public.
This is why Go is spreading - good middle ground for productivity and performance, easy to pick up, seemingly bright future, Google backing, good documentation (no doc is perfect - Sphinx is frequently not great, Javadoc is terrible, Doxygen is meh, MSDN is overgrown with cruft - but Godoc is decent), etc.
> Similar performance to Java but with much less code, better interoperability, and a more robust standard library.
I disagree on all points.
1. The similar performance to Java is usually in small benchmarks. Once the program grows, the difference in quality of the GC and code generation starts to show (in favor of Java, of course).
2. Less code? I've seen Go programs in the wild that are indeed shorter than their Java counterparts, but that's almost invariably because the Go code is less "careful" (less attention to error handling, logging, monitoring, maintainability etc.) But when you compare programs that actually do the same thing, I find that the code size is similar, with Go winning fewer characters and Java 8 fewer lines.
3. Interoperability is, IMO, much better in Java, where you can interoperate with both C as well as JVM languages (including JVM ports of Python, Ruby and R); for Go it's just C.
4. As to the standard library -- especially when it comes to concurrent data structures -- there are few if any languages out there with implementations of as high a quality as Java, and the difference between the variety and quality of available third-party libraries is beyond comparison.
All in all, I find Go and Java to be quite similar, and the choice between them boils down to the runtime. I reach for Go when I need small command-line programs where JVM warmup is unacceptable and static linking to a native executable is a plus, and Java for long-running, "important" server apps, where top performance and deep monitoring (and sometimes hot code swapping) are necessary.
I agree that Go and Java are very similar indeed. Without goroutines, Go would be little more than a rewind of Java to a moment before J2EE happened. A major motivation for both languages was to take dangerous weapons out of the hands of mediocre programmers (or programmers made mediocre by being forced into large and ever changing teams).
But I have to disagree on performance. What kills Java performance is memory consumption and the way memory is laid out. It has always been the number one killer. It killed Applets. It killed Java on the desktop. And it makes Java a bad choice for most server side infrastructure. I predict that Go is going to wipe out Java for that latter kind of task within 5 years if not less.
Memory consumption is what constrains performance for most tasks. That's why most benchmarks are entirely irrelevant.
Java's creators made two huge mistakes:
1) Throwing out structured value types, which is the root cause for Java's insatiable appetite for memory. 20 years of garbage collection research were unable to compensate for the damage done by not having structured value types.
2) Throwing out so many (supposedly too complex and too dangerous) meta programming features that everything meta had to be built around the language. Trading complexity at the language level for a much more unsound form of complexity on the runtime and tooling side is what caused the J2EE disaster.
Go's creators avoided the first mistake but they are busy repeating the second one with even greater determination.
I disagree with your analysis. Value types are being added (that's what the HotSpot team is busy working on) with relatively precise control over memory layout, and their negative impact only became clear when random memory access became more expensive (in relative terms). Still, Java performance isn't "killed" for the simple reason it handily beats Go, and often enough beats C++ in large, multithreaded programs. So if this is what Java performance looks like when it's "killed" by lack of value types, I'm very hopeful at the prospect of adding them, because Java will have no competition then, let alone Go, which lags behind even now. Large, big-data analytics software (like H2O) written in Java achieves 100% of maximum hardware performance ("Fortran speed"). The trend is that more and more performance-sensitive applications are transitioning from C++ to Java. Also, Quasar adds true fibers ("goroutines") to Java.
Now, I don't know exactly what you mean by memory consumption, but if you mean total memory consumption by the JVM, that's a feature of the GC chosen, which usually trades extra RAM for added performance -- it certainly does not negatively affects performance.
As to point 2, I think that was Java's greatest decision (which Go can't replicate), because the malleability of bytecode afforded by class loaders and agents have made the JVM extremely flexible while at the same time extremely performant. At the language level, annotation processors -- really, AST transformers -- are extremely powerful (see the new pluggable type systems for Java 8) and not easy enough to abuse by under-qualified developers. If you want a more powerful language, the JVM offers quite a wide selection, which is another thing Go cannot do.
Then assign the JVM less RAM and suffer worse performance -- it's often a simple tradeoff you can make when launching the app. It would still likely be faster than most other choices (certainly all other GCed environments).
Did that (reduce heap size), ended up having 30 minute GC loops. Implementation in another <unnamed, new> language decreased memory usage rather radically.
Well, I guess I could have also done Structure of Arrays implementation and additionally avoided 'new', but it's not Java anymore at that point, is it? Easier to just write it in another language at that point.
Generic branchy business logic type code is indeed about as fast in C++ as in Java. But that doesn't hold for all types of software.
How fast is Java native floating point math vs. C++/intrinsics? Last I checked, C++/intrinsics seemed to have 2-8x lead. What about sorting? Last I checked C++/intrinsics sort seemed to be about 20-50x faster versus Java.
Fully agree with 1), specially if you look at some alternatives back when Java appeared into the scene:
- Oberon
- Oberon-2
- Active Oberon
- Component Pascal
- Modula-3
All with GC, value types, generics (just Modula-3), system programming capabilities and compilation to native code in their reference toolchain (Oberon variants also had JITs).
But now we have Java, with large installed bases, very mature tools, and latest by Java 10 that mistake might be fixed.
Asking the wrong person for that one - I just use Vim ;-)
I do use a few of the Go scripts (https://github.com/robmccoll/dotfiles). There are some really nice ones (continuous build, syntax checking, continuous testing, etc), but most of what I work on has longer build times due to linking in a good bit of C.
It's worth noting that a number of JetBrains employees seem to be working on this plugin every day [1]. I suspect we'll see a proper JetBrain Go IDE before too long.
I use Eclipse with the Goclipse plug-in. I still have the command prompt on the side for some stuff but the plug-in covers a fair bit of what you need to get done...
When you first look at go, you might think, "gee, this language sucks. What is this? A compiled language with pointers, structs, that's not object oriented? And it has garbage collection? WTF. Okay, so this is meant to replace C/C++?"
Nope. Go and Rust are almost entirely orthogonal in their best use cases.
Go is a langauge for people somewhat comfortable in low level langauges that want to high level things in a somewhat performant way. The older alternative is probably python.
Rust is a language for people moderately comfortable in low level languages that want to do moderately low level things in a complex way. The older alternative is C++.
C is a language for people very comfortable in low level langauges that want to do very low level things. The older alternative is assembly.
Go and Rust are almost entirely orthogonal in their best use cases.
I disagree. I think in many (but not all) use cases, Go and Rust are direct competitors. However, they embody a different philosophy. It's New Jersey (worse is better) vs. MIT style all over again.
One can read Gabriel's[1] descriptions of both styles and they map exactly to Go and Rust. Given Go's heritage this is, of course, not surprising.
Agree. Go is made for web/distributed programming: builtin concurrency, battery-included library with great support for http utils, including a production-ready server and other stuff like RPC. And the distribution is just so easy.
For a Pythoner, Go almost solves all my pain points for python, notably the lack of support for real multi-threading so that for long running task we have to turn into processes based 3rd party library like celery/rq with a message queue as intermediate layer, while comes with a relative familiar favor of syntax.
Go is a pragmatic minimalist language, with limited but cohesive set of features that stays at its core. I don't know rust very well, but from what I gather from HN, looks like that it aims to be a more comprehensive solution rather than 'opinionated' one in case of Go.
I read that back when it was first published, believed it, and built a few teams using it's philosophy. I've also been at a few big companies that haven't followed it's direction. In my experience it doesn't work unless you have a small team and a small company and even then you are often left with fickle programmers who leave when the next fad comes along because your company can't justify rewriting its systems with the latest fad tech.
Wouldn't you say lisp was the opposite of a fad at the time when PG wrote that article?
It's been a long time since I read it, but in Microserfs, the classic "VC" or "Money" guy describes how you can drive around the Valley on a Sunday and predict the success of companies by how many cars there are in the parking lots.
Most conventional wisdom these days is that if you have to work 80-90+ hour mega weeks you're doing it wrong, but I think that the analogy might be that if the team is using an interesting technology stack, there's a higher chance the engineers there are of higher caliber (as they're interested in pushing themselves to use a newer technology.)
"but I think that the analogy might be that if the team is using an interesting technology stack, there's a higher chance the engineers there are of higher caliber"
My experience doesn't support this for a few reasons. One of the main reasons is what I alluded to before, the people interested in "interesting" tech stacks are fadish, even if the tech isn't. In 2-3 years, your "interesting" tech stack becomes "uninteresting" or worse, a bad idea. I remember when java was "interesting" then it was php, then python, then ruby on rails, then scala, clojure, haskel, then backbone, then node.js, then angular, then react, then go and rust, then ....
As a big fan and daily user of Go, I agree with this. I've used it to solve concurrency problems or problems wherein I could quickly replace a slower long-running process in Python with something just as simple but more performant.
I see a lot of codebase refactoring in Go that doesn't really need to be written in Go. I'm not sure what the impetus is if not to just do it or to reposition a staff's codebase. The argument for these sorts of things should be better expressed, in my opinion. "We moved to language X because ..." should be a simple 1-line explanation, even if it is indeed "we want to work in the language or attract a certain type of developer."
I recall reading something similar wrt Scala & Twitter. At the time Twitter really needed to scale up they chose Scala and had the effect of attracting people who wanted to work at a company because they used Scala. I can't find the link however
>The first thing I would do, after checking to see if they had a live online demo, was look at their job listings. After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried.
I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value, which many would argue is the case for Go (even though Go is generally a nice language).
Granted, you did put "cutting edge" in quotes, but it's still worth stating explicitly, Go isn't cutting edge, and neither are Ruby, Node.js, NoSQL, or most of the other technological rehashes ignorant developers get excited over.
These people would never pay any attention to old, boring technology like Lisp or Smalltalk or even (given the success of NoSQL) ACID-compliant relational databases like PostgreSQL. But repackage some of the good ideas from them and pass them off as your own, fire up your hype machine, and these clueless devs will eat it up.
Why, look at the cult-like following Matz enjoys for creating a half-baked Smalltalk and Perl knock-off.
90% of the value created by using Ruby is because of the exemplary tools put out there by the people IN that cult-like following.
There's a lot of programming languages and programmers totally focussed on application performance as well as squeezing the limits out of their hardware. The ruby community is focussed on developer performance and squeezing the limits out of their people.
If more languages and communities spent as much time on streamlining everything around time to market, Ruby would not be nearly as big as it is. Because most are not focussed on that, Ruby stands out significantly when it comes to getting big things done with fewer people.
That's why so many start ups use it. That's also why so many start ups eventually come back and refactor the slower parts of their system with something like Go...but the important thing is that they got to market, built enough of a customer base and revenue stream that they can both a) afford to refactor and b) need to refactor because so many people are using it.
Building a language and tools around people rather than process is a unique undertaking. Because of that Ruby and Matz deserve every ounce of credit that they get. Doesn't have to be right for everything, but what they try to do well they do exceptionally well.
That's true of any very big organization and news companies are very compartmentalized and segregated. I recall quite a few different dev groups not including R&D at the Times that used dissonant tools - I didn't even see RoR listed, which is used extensively in news teams within the Times.
The ones who actually know their history and consequently, when they hear "NoSQL," they think of IBM's IMS and run as fast as they can in the opposite direction.
it actually makes good business sense for a new organization to capitalize on these decisions. It is similar to a consulting agency - show off the use of a cool/upandcoming programming language or approach and earn more eyes or potential customers. Side benefit- the team also gets to improve their skills and enjoy the job more.
Are Go servers at New York Times or other companies impacted by the Golang GC pause? I am learning Golang and want to know if Golang GC pause is a significant issue in production.
From what we can tell, yes. But, according to our Google contacts, we're among the top performers, most parties taking part in RTB perform a lot worse.
