It's a little more complex than that:

If you disable all js, there is no delay. If you specifically block amp js, there is a delay. There could certainly be a shady reason for this, but the justification makes sense: one of the goals of AMP is to have, basically, a single, correct, initial paint. That's why images are statically sized and amp sticks placeholders there until the real images load. The js delay exists to allow the site to fetch the js and use it to correctly render the initial paint, even on a slow/flaky connection where the js takes a bit to fetch.

The delay is css based not js based, so why would blocking all js affect it?

If you tell the browser to disable execution of all <script> tags, it will instead evaluate the <noscript> tag which contains some additional CSS that disables the animation.

Ah, I didn't realise/remember it has a noscript block, it's been a while since I dug into the issue originally.

Sites which optimize their pages via https://www.npmjs.com/package/@ampproject/toolbox-optimizer have the initial layout performed server-side instead of by javascript, and thus the CSS Flash-of-unstyled-content protection is removed. amp.dev runs this optimization, along with lots of other sites.