Too bad then that desktop Linux is so easy to pwn.

Obtuse gotcha. Trust and security are two different things. And Linux is as secure as anything else against physical attack.

edit: local, not remote attack

When you write, "physical attack", what do you mean? Laptop theft?