> Solutions that write the received file to $TMP and just run an external tool (or, say, use sendfile()) don't have the option to abort after N decompressed bytes

cgroups with hard-limits will let the external tool's process crash without taking down the script or system along with it.