I’ve noticed that too when the scroll back buffer is particularly long. iTerm seems to then reflow from the beginning, which can take a while.

I keep using a lot of fresh tabs all the time and iTerm resizes instantly via Rectangle for me.

Ghostty was a bit more sluggish in resizing on an Intel Mac than iTerm in the current latest versions.