Good point. This is the reason I never listened to my math teacher, his material was too basic and too abstract for me to get motivated, I couldn't see myself using that knowledge in the short term. I went to art school, and only later I realized I was actually clever enough to program and enjoy it; all it took was a high level language like ActionScript 1 (and to put credit where it's due, Flash allowed a nice smooth transition from manual design to code). After that I worked my way down to Java, Objective-C, and more basic C concepts (although Ruby/Python are the owners of my heart).

Ironically, this book looks interesting to me because it would now fill the basic gaps. I've never actually needed to understand binary numbers or endianness - it's bitten me in the ass a couple times but nothing looking for help online couldn't handle - but today those higher layers of knowledge have solidified enough in my brain that I'm ready to actually enjoy learning these basic concepts.