I am slightly confused. How can they have use their "hash map of popular passwords" to diff against your table if they do not have the "pepper" used originally for bcrypting the passwords?

Pre-computed mappings of popular passwords (rainbow tables) aren't really used nowadays due to parallelization being more cost effective, but the idea in both cases is that you want collisions with known values: the popular password hashes in the case of the rainbow table, or the computed values in parallel enumeration.

If the password hashes each have a stored unique salt (bcrypt will), you have to compute the hash per salt per password that you test. Instead of computing the hash for "password" (+ stored work factor for bcrypt) to be X, and checking all database entries for X, they instead have to calculate each hash per entry.

For a table of 1000 users, it would take around 1000 times as much work to determine the users with "password" as their password. If you just wanted to target a single user though, the salt doesn't really matter for enumeration (though if you were using rainbow tables, you likely wouldn't have those specific hashes computed).