It may be situational, too. I have a family, and when I get the house to myself for a day, it definitely improves my mood and sense of calm. It's very restorative for me.

I can be in my office away from the family, but it's a completely different feeling than being actually alone in the house. Doesn't matter what I'm doing, either. Working, reading, playing, watching TV, etc. It's a very different feeling.