Some systems restrict you to alphanumeric passwords (generally for no good reason). If you reuse your password across systems (or you have a formula for creating passwords) then you are less likely to use special characters in case 1 system requires a different one.

Additionally, entering symbols on a phone keypad or touch screen is usually a little harder.