> REQ/REP would pretty much always block forever if the peer goes away

unless you tell it to do otherwise, that's exactly what it will do. The guide covers this: 4. Reliable Request-Reply Patterns