Reading this causes me to experience déjà vu; years and years of reading stories and watching presentations about someone struggling with GC in the JVM. It's happening all over again with Go. The same 'discoveries', the same trade-offs, the same discussions about hardware resources, the same 'concurrent mark and sweep', the same 'more to do' conclusions. You could replace every occurrence of 'Go' with 'Java' and it would probably go undiscovered.
Maybe it's all worth it and this is how developers are supposed to spend their time, but it's no longer interesting to me.
It's because GC is a bad idea that's had 30 years of good research thrown after it. Advancing GC is building a faster horse instead of stepping back and building a car. It's time to move on, and I'm thrilled to see modern languages (Swift, Rust) abandoning it and focusing on building more intelligent compilers.
The goal shouldn't ever be to make the world's best GC, it should be to create the world's best way to elide lifetimes so that developers don't have to think about memory management. GC shouldn't be a goal, it's a technique for solving a problem, one of many that we should explore.
Seems like the opposite surely? We should be developing languages that more succinctly address the problems we humans are trying to solve not book keeping for the computer's hardware (that should be the computers job!).
I think you missed the bit where he said "more intelligent compiler". The compiler is the bit that does the bookkeeping, only in Rust (and evidently Swift--I haven't played with it much) it's done statically, ahead of time rather than at runtime.
That said, I think Go is a much more practical language than Rust for most problems. That said, I'm still very excited about Rust.
These are different things. A sufficiently smart compiler is a hypothetical compiler that could theoretically optimize a high level language so that it could be faster than some low level language. This isn't what we're talking about here--we're talking about the concrete ability of the Rust compiler to preclude certain classes of errors.
Yeah, that's exactly what I was trying to say. Rust does all the bookkeeping at compile time, Swift keeps a lot of it at run-time although the compiler can easily optimize away a lot of lifecycle stuff too when it's in scope, so I assume it either does or will.
I agree that Rust likely does not have the be-all answer to automatic memory management, though what I love about it is that they're pushing the boundaries and getting people thinking differently about memory management.
> I agree that Rust likely does not have the be-all answer to automatic memory management, though what I love about it is that they're pushing the boundaries and getting people thinking differently about memory management.
Me too. I intend to use it for more of my hobby things, but Go is currently the best fit. Eventually I imagine Rust will pick up some decent GUI libraries or at least get decent editor support (vim-go is lightyears ahead of YCM+racer) and I'll be able to afford to justify using it more.
What he said is that the goal should be that developers need not handle memory manually. GC is one technique to achieve that goal, and the one that has been the most successful so far, however we should not equate automatic memory management and garbage collection: other techniques could offer an as good or even better experience if we took the time to explore, develop, and refine them.
GC is also required for persistent data structures which makes it a must have for languages where immutable data is a fundamental strategy for handling concurrency.
> Reading this causes me to experience déjà vu; years and years of reading stories and watching presentations about someone struggling with GC in the JVM. It's happening all over again with Go.
It's because GC is an area full of tradeoffs, and despite popular belief, the HotSpot GC is really good. In fact, I honestly don't know of any way to improve on the HotSpot GC for general-purpose use (i.e. throughput/latency balancing). HotSpot has a generational, concurrent, compacting GC; allocation takes 4 or 5 instructions (really!); the compiler has SROA to aggressively optimize out allocations where unneeded.
Look up the Azul Systems pgc. That is more or less the holy grail, but it is patented out the wazoo, and is a commercial implementation only.
Also, the jrockit jvm (which was from BEA and was purchased by Oracle) is actually quite a bit faster than hotspot and easier to introspect (lookup jrockit mission control) than hotspot. I suspect eventually they'll merge however.
> Look up the Azul Systems pgc. That is more or less the holy grail, but it is patented out the wazoo, and is a commercial implementation only.
That's what I was alluding to in the parenthetical. According to the paper, C4 trades off a significant amount of throughput for reduced latency. That's what you want for many applications, and C4 is a great advance for those apps, but throughput is very important for most workloads, so HotSpot's GC ends up yielding a good balance.
It's not perfect. Azul's C4 does a lot of work in read barriers, so code that looks intuitively like it should be fast can end up causing "read storms" that bog the code down.
C4 never pauses, and that's impressive. But there's no free lunch. The work the GC would do when the app is paused is sometimes being simply done by the app threads instead.
I heard the read storm problem has been solved by Shenandoah, a GC developed by Christine Flood (who was in the original GC G1 team at Sun in 2001). It is under the RedHat umbrella and should be merged in OpenJDK [1].
Shenandoah uses a forwarding pointer in each object, adding overhead but limiting the problem only to write barriers. Here is Christine commenting on Azul vs Shenandoah [2]
From the talk: average pause is 6-7ms, max is 15ms, and the talk is one year old.
She hints at further developments in a version 2 which would make it entirely pauseless.
She has made another talk at RedHat's DevNation conference a few days ago, but they just won't put the video online arg!
On hotspot: There are two bits in the header of every object. This is enough for an object that's never been used as a contended lock, CAS operations on the header can be used to handle the locking and that's that. As soon as you actually block on it, a 'real' lock is created (you can't get around the need for a list of threads to wake up as the lock is released) and the header grows to accomodate it. The process is called 'monitor inflation'. At a later date this might be cleaned up by a 'monitor deflation'.
There's a certain amount of work that has to be done for GC, and that work is going to be done somewhere. The question is just what trade-offs you want.
Don't want compacting? You'll pay for it in allocation.
Don't want pausing? You'll pay for it in application threads.
Precisely, and this is what is so often missed in these discussions. Most of the time, when you see claims of GC silver bullets, there's some hidden downside that's being papered over. Latency wins (i.e. "max pause time" or whatnot) tend to be throughput losses. Less copying results in more fragmentation. Value types can result in more copying, reducing performance over pointer indirections through nursery allocations. And so forth.
I don't know that anyone disputes this. The discussions I participate in don't deny this; they mostly talk about whether or not the tradeoffs result in a net gain (if you sacrifice a little from the minority of cases to gain the same amount in the majority of cases, you do indeed have a net gain).
> Value types can result in more copying, reducing performance over pointer indirections through nursery allocations
Having value types means you can pass by copy, but it also means you can allocate on the stack and pass by reference--in other words, you get performant passing without involving the GC.
Nearly all the work you allude to is done in other threads - which indeed consume machine resources (CPU cycles, memory bandwidth). If your application does not burn all cores/bandwidth then the GC work is all done on the idle/spare machine resources. At the limit though, indeed you'll have to slow down the Application so that GC can play catchup - and bulk/batch stop-the-world style GC's require less overhead than the background incremental GCs
Cliff
My understanding: C4 does not pause but the JVM still does for various other reasons and a part of the work on Zing has been forcing down those pause times too.
Sorta kinda all of the above.
C4 the algo has no pauses, but individual threads stop to crawl their own stacks. i.e., threads stop doing mutator work, but only 1-by-1, and only for a very short partial self-stack crawl.
C4 the impl I believe now has no pauses also.
HotSpot the JVM has pauses, and yes much Zing was on forcing these pause times down.
Cliff
Isn't it also because Java and/or having GC makes certain patterns easy although they should be hard? At least my limited experience with Java, from writing a system which dealt with millions of integers, is that Java really wants you to use ArrayList<Integer> instead of the GC-friendly int[].
The problem with Java is that most things are a pointer, which means the GC has to deal with it. Go on the other hand allows the user to specify which things should be a value and what should be a pointer, which significantly decreases stress on the GC.
C# has something called value types, and while this helps (and Java is working on implementing something similar for Java 10) it's not as flexible as Go, where users can decide this at whim instead of specifying it in the type.
Java doesn't "want" you to use ArrayList<Integer>, that's merely more convenient if you need a dynamically sized array and don't want to do the resize yourself.
But the JVM folks are adding support for ArrayList<int> to the language, with the efficiency you'd expect from it.
Value types would hopefully get a big help in terms of getting the JVM GC better once the SDK and popular libraries full utilize it, both by reducing memory pressure and making the heap more GC friendly.
Fun stuff I've been doing with the H2O project is basically using nearly-pure Java (some Unsafe) to hold onto numbers with better efficiency than e.g. int[], and giving out an easy-enough-to-use API for writing parallel & distributed code over an Array-like API. i.e., feels "almost like an array", and "for-loops" run at memory bandwidth speeds and also parallel and distributed.
Cliff
Are the number of garbage objects generated by idiomatic Go and idiomatic Java comparable?
My guess is Go implementation will produce an order of magnitude less garbage.
In Go, an array of structs (= objects) is just one object.
In Java, an array of objects is array object itself plus one object for each value in the array. Except for elementary types, like bool, int, long, etc.
I completely agree and with so much tuning required in past frameworks for their GC it makes me wonder why more don't simply adopt the C++ / Rust models of resource management.
I remember way back when people said you couldn't use the JVM for real time applications because of the GC pauses but it's been improved significantly since then and now all the same topics are coming up with GO.
Because the C++/Rust way of memory management is better for some things, but worse for others. I've worked with several different projects during my career, and not once did we require manual memory management. A GC based language was simpler for us to use, and the few times we had problems with GC, they were possible to overcome by writing better code, as is the case with any language.
This is not to say that no project benefits heavily form C++/Rust. But I would argue that for many, GC is the best trade off.
I completely agree that explicit memory management (I wouldn't call it manual) in the C++/Rust way is a cognitive overhead you don't want for a great deal of the software work - perhaps most of it.
But there are definitely projects that require explicit memory management, and it's not just games and realtime software. Often high-performance backend code in Java and Go just end up using object pools instead of reallocating objects, just as the OP described.
With Go specifically we've seen the rise of fasthttp, which just adds completely manual memory management in the 90's C++ fashion. Want to create a new request object?
I'd argue that one of the biggest differences between golang and java is not technical but cultural. That is, the golang idioms and thus the std libs are quicker to reach for things like object pools and other performance "hacks". Even the std http library uses an arena in golang.
Similarly, high performance Java libraries like the Disruptor, SBE or Chronicle look very much like C code.
Personally, that doesn't bother me, as it allows you to write your hot path and your non-optimized path in the same language with the same tooling.
says the guy who has split JVMs across processes for performance and contemplated doing it per core
I'm not sure where you got manual memory management from. I was strictly referring to RAII. Manual memory management is such a pain but the C++ and Rust ways are very similar with RAII.
I usually refer to everything that isn't GC as 'manual'. RAII is an abstraction on top of manual management, you still need to decide what type of pointer/lifetime the allocated object should have, making it manually managed, IMHO.
RAII of course deals with more than just memory, but in a thread about GC I assumed it was memory management you referred to.
>Maybe it's all worth it and this is how developers are supposed to spend their time, but it's no longer interesting to me.
Unless you design and implement GCs yourself, it's not supposed to be interesting to you anyway. It's just something that will benefit users of the language, not something to excite them.
If it's not supposed to be interesting, why do so many that have found themselves running up against the limits of GC in their chosen language/platform/implementation end up writing epic blog posts that represent months or years of work and/or presenting their hard one solutions at conferences? There certainly seems to be a lot of people that end up having to be very interested in solving their GC problems once their systems grow to non-trivial size, and they all seem to be relearning and resolving the same set of problems.
>If it's not supposed to be interesting, why do so many that have found themselves running up against the limits of GC in their chosen language/platform/implementation end up writing epic blog posts that represent months or years of work and/or presenting their hard one solutions at conferences?
Because they care about improving actual, existing, languages, with actual, existing, ecosystems, not doing cutting edge academic memory management research.
>and they all seem to be relearning and resolving the same set of problems.
So like architects relearn and resolve the same problems, about building bridges, skyscrapers, condos etc -- instead of designing some new structures to replace them?
The cruel irony here is that simplicity was a foundational goal and major rallying point for Java. Here's a website from 1997 describing it: http://www.cafeaulait.org/course/week1/16.html
Ignoring the last part that obsesses over the glory of OOP, replacing Java with Go in that page is... pretty spookily familiar!
> I sincerely hope Go does not go in that direction. The 'less is more' approach is so far very strong among the Go steering committee.
There is no "less is more" approach in Go. It's more like you can't write something really complex in Go so people use it for trivial things like servers that do almost nothing aside from de-serializing JSON. Try write a large LOB app in pure Go or a fully featured CRM. And see if you can get away with "less is more" when you need to reason about complex business rules, data validation, complex routing, mapping RDBMS data to values, and what not. "less is more" is a mirage. Go short comings will show up pretty fast.
All you're really saying is that Go is not great for web apps. I admit, it is not. So is C. I would not write a CRM in either of these languages.
At codebeat (codebeat.co) we use Go for our backend - very CPU-heavy, complex static analysis workflows. Our frontend is in Rails which is not ideal but probably the best bang for the buck for an early stage startup. This is the beauty of having many tools to choose from.
> All you're really saying is that Go is not great for web apps. I admit, it is not. So is C.
C is 30 years old, so it has an excuse. Go has none. The fact that it's extremely difficult to write a classic, complex webapp in Go is a proof that this language has serious flaws.
It is just as easy as doing it in Rust or Swift - both being thoroughly modern languages. It's more about the lack of comprehensive frameworks than the language being somehow flawed. I used Go as a backend for mobile apps and it was OK but where it really shines is the kind of workload we do when analysing source code: where you need excellent performance and low memory footprint, all that while keeping the code readable.
> It's more like you can't write something really complex in Go
Have you recently checked out the bigger projects that are currently being written in Go? You'd be surprised...
> Try write a large LOB app in pure Go or a fully featured CRM.
Woah. Have you tried doing that in C, C++ or Rust? Has anybody? Every language has it's strengths and weaknesses. Sure it's possible to do so in them - but is it a good idea? Not necessarily. I'm not going to write a database engine in Python - but we have timeseries databases being written in Go.
> Go short comings will show up pretty fast.
Every language has shortcomings. Go's major thing seen as a shortcoming is the classic "lack of generics", which arguably is true to some extend - but not it's GC. The thing is - Go's strong points have become clear long before these shortcomings you're talking about. The entire ops-space jumped on it because it solved a few problems plaguing their tools: memory overhead, slowness, dependencies, hard to make portable. Pretty much every major new project related to infrastructure is created in Go.
One of the biggest attractions of Go is it's ability to create programs that perform a lot better than the same thing written in Ruby or Python, which then again allows developers to undertake more ambitious projects.
Maybe it's all worth it and this is how developers are supposed to spend their time, but it's no longer interesting to me.