Well, okay. But the result of one group of people being governed by another is also war.
We have different levels of government for a reason.
EDIT: To other points occur to me: 1) I've never found Hobbes very compelling. And 2) If "these other people affect me" is your (not you you, but the general you) idea of a casus belli, you will perpetually be at war no matter how large or small your government.
For a primer on the philosophical reasons behind government, I recommend _Leviathan_ (1651) by Thomas Hobbes.