But aren't movies rendered to the screen?

My understanding is most smartphones at least have hardware decoding for H264.

Exactly, a GPU needs to have specific hardware (a dedicated ASIC) for H264 acceleration. It's not accelerated by the usual GPU hardware, which is sufficiently general-purpose that an accelerator running there would not be codec specific.