The general principle underlying all of these situations may in some ways seem like common sense, but it's worth repeating: one side can only know the state of the other side if the other side tells it; and vice-versa.
A cool thing HTTP/2 provides is PING frames, which allow clients and servers to check if connections are still alive. In case your application can't turn on TCP keep-alives, you can enable it at the application layer. This allows both sides to know if the connection is still around and active, and avoids NAT boxes from removing long lived connections. Rather than wait around 5 minutes, or tune your system, you can control how quickly dead connections are discovered.
It may not be the first but it is a reason to upgrade from HTTP/1.1. PING frames have other uses too, like being able to estimate connection latency in an application, rather than trying to read it from the kernel. This can be used to load balance across connections, or estimate the bandwidth delay product of a connection. PINGs have a lot of uses; the new thing is being able to control it from an app rather than it being OS specific.
tcp keepalive interval defaults to 2 hours on most systems, that's not sufficient to keep NAT mappings open. So besides enabling keepalive you also need to adjust the interval.