A question to the guys who had severe withdrawal symptoms: did you drink it with or without sugar?

I personally don't put sugar into coffee or tea and have quit cold-turkey a few times without any issues, so I wonder if the headache and stuff might be somewhat related to sugar withdrawal, not just caffeine. Ideas?