The issue is while your system volume can be the same, media volume varies immensely. Listening to one album, it could be all relatively low and gentle. The next one could be noise that blasts your ears but you don't really think about it due to focusing on something else.

Sometimes I'll try adjusting my volume down to half and notice everything still sounds fine. It's a sign that I was listening to something way too loud without even realizing it.