Self-defense ends when the immediate threat is incapacitated. Ender fights on past that point. The umbrella of "self-defence" does not cover all that he does.
That's kind of the point in the book - they chose Ender because he would finish every fight definitively, conclusively and not waffle. Its actually one of the themes.
For a weak, picked-on child it was Ender's only defense in a violent world (from a child's point of view). He could not survive a truce that was then violated by the opponent. It would only give them the opportunity to surprise him later. He had to win on the 1st encounter.
The interstellar conflict was similar - the enemy attacked first, had a superior war machine and with time could overwhelm humanity.
Anyway, the point of books like this is to explore themes like that. They don't dictate a viewpoint to the reader, they provoke discussion and enable insights,if properly introduced to the juvenile reader.
