I didn't know the PS3 did complete emulation of the PS2 at all. The earlier PS3s that supported backwards compatibility actually contained the PS2 hardware (the first gen had both the cpu and gpu, and the second gen PS3 had the GPU and used emulation for the cpu).

I know that there are a lot of PS2 games in the PSN store, but I was under the impression that those were actually recompiled for the PS3 and did not use emulation. Are those actually emulated?

The early ones were ports, they were recompiled and enhanced with higher resolutions and/or framerate, trophy support, etc... They're titled "HD Collection" or "HD Remastered".

Many of the later ones are actually emulated. I think they're called "PS2 Classics" or some such.