That's really weird and I have no idea why it wouldn't be working for you.

I even went as far as to spawn the ancient version of powershell included with Windows and pwsh 7.3.4 without a profile and tab completion adds the ampersand. I do import the PSReadLine module though in my profiles so maybe that is providing the functionality and I was wrong about loading an empty profile.