Here at The Agency, we often work collaboratively with partner agencies who are responsible for the analytics & reporting for a client website. Without fail, they pick the most impressive numbers to present, not the most reflective of "business success" (which, admittedly, can't always be directly related to sales figures, but you can usually do better than "overall page views month over month"). This isn't their fault - it's the sort of stuff mid-level brand managers want to take to the CMO to say, "See, it's working! Please increase my budget." And the CMO doesn't mind much, either, because he can go to the board and say, "See, it's working! Please increase my budget." And the board likes to leave their meetings happy, so they generally don't mind good news... except when that good news doesn't match what the CFO is presenting for their revenue.
Measuring the wrong thing is endemic to technologists because we have the opportunity to measure everything! Drucker once said, "what gets measured gets managed." That couldn't be more true. If you optimize (manage) for monthly actives and count the signup visit, you probably don't know if people actually like what you got, or if you're just shouting loudly enough for them to glance at you. I'll repeat my usual mantra: measure as close to the cashflow as possible, then logically back your way out from that to understand your funnel. If you can't put a dollar value on an action (10% of pageviews turn to signups, 25% of signups become active users, 60% of active users become paying members, paying members are worth X, therefore a pageview is worth Y), it might not be worth measuring.
Quasi-apropos: what makes Facebook so impressive to me is not that it reaches 51% of Internet users, but that the average one views over 1,000 pages a month[1].
this is one reason my favorite favorite metric is number of customers: the number of people who have spent money on my product/service during some time period. pretty hard to fudge that number
Great post. Minor quibble, I would say that "the unique cookie framework overestimates the actual numbers by 33% (40% instead of 30%)," rather than the 25% number given in the post.
Measuring the wrong thing is endemic to technologists because we have the opportunity to measure everything! Drucker once said, "what gets measured gets managed." That couldn't be more true. If you optimize (manage) for monthly actives and count the signup visit, you probably don't know if people actually like what you got, or if you're just shouting loudly enough for them to glance at you. I'll repeat my usual mantra: measure as close to the cashflow as possible, then logically back your way out from that to understand your funnel. If you can't put a dollar value on an action (10% of pageviews turn to signups, 25% of signups become active users, 60% of active users become paying members, paying members are worth X, therefore a pageview is worth Y), it might not be worth measuring.
Quasi-apropos: what makes Facebook so impressive to me is not that it reaches 51% of Internet users, but that the average one views over 1,000 pages a month[1].
[1] http://www.google.com/adplanner/static/top1000/