It is pretty small, especially when you look at lenses. When I go on short vacations, I can fit my M43 Olympus and three lenses into the same carry-on backpack I simultaneously pack all my clothing, electronics and bathroom accessories in. I've owned an APS-C before, and realistically speaking you'll have to pay extra for check-in luggage and wait at baggage claim if you're going to bring such a camera.
This is obviously all nonsense if you're enthusiastic (or professional) enough, but I struggle to bring my camera along as it is.
That's only true when comparing lens-less cameras, which isn't that meaningful when lenses are required to use the body. An X-E3 with lens isn't a similar size to an E-M10.
Panasonic's GM series is all but abandoned, and the E-M10 is hardly small compared to a Fuji X-E3 or Canon M50.
To me, that always felt like M43's Achilles heel; it promises to be small and light, yet most kits actually end up just as big as APS-C.
I still love my GM1, though.