If you actually want to understand the "math behind it all," do yourself a favor and read the first three chapters of _Probability Theory: The Logic of Science_ by E. T. Jaynes. Jaynes builds, from the ground up, probability theory as an extended logic that allows you to draw inferences from incomplete and uncertain information. In this logic, a probability represents a degree of belief, and Bayes' theorem becomes a rule for updating prior beliefs in light of new evidence. From this basis, Jaynes recreates the classical probability theorems (e.g., sum and product rules) , giving them clear interpretations.

If you want to understand this stuff for real, it's hard to beat Jaynes. (Plus, he uses a robot mind (!) as a recurring expository device. That alone is worth the price of admission.)

[1] http://bayes.wustl.edu/etj/prob/book.pdf