Recovery numbers are recovery telephone numbers. If they sim-swap you, they have your telephone number. The end. Virtually all email accounts and online services tie themselves to your damn telephone number, and there are tens of thousands of people in each phone companies who can move your number.