The first MP3 I downloaded was from IRC and played back on the Fraunhofer Institute MP3 player. Old timers will remember that if you caused your screen to redraw at all, say by scrolling a window, the audio would skip. The first song? Bob Dylan - Stuck in the Middle With You… which is actually by Steeler’s Wheel.
An F-serves on IRC. Much respect to the glorious bastards running FTPs and F-serves off their dorm ResNet connections. The rest of us poor slobs on dial-up could actually max out our downstream bandwidth.
There were also quite a few websites that just straight up hosted MP2s/MP3s for public download. I can't recall domain names but I do remember .se and .ru being common tlds.
I couldn't read a single word of Swedish or Russian, but my friends and I figured out how to navigate to the music downloads.