I guess the non-secret parts of the card number can totally be cross referenced with others merchants you've shown the card, but an online purchase very often also has a name and address so your identity is already out the window.

Also fraud detection systems will often track the type of purchases associated with a particular card number, to detect anomalies. So I suppose your privacy is somewhat protected from that, but the e-commerce sites probably already know who you are.