It's unfortunate that the person who wrote this article didn't consult with someone more informed about the online advertising world.
As someone who has been in the belly of the beast, I'm going to try to summarize the various ways that this article is getting it wrong.
* Most Yahoo Mail ads generate revenue based on impressions, not clicks. So Yahoo isn't directly making more money by grabbing these "random" clicks.
* Where clicks do matter, major ad platforms including Yahoo throw away a LOT of clicks as fraud, and accidental clicks tend to disproportionately get thrown out along with them.
* In general, random clicks are considered a real PITA for major ad networks, as they confuse the heck out of ad optimization. While small players do tend to soak up that revenue, the big players really, really hate the phenomenon because it makes them far less efficient.
* It turns out what most impacts ad effectiveness online is whether people actually see the ad. An incredibly number of ads are just never seen by the audience. As a consequence, a good publisher will try to find locations for their ads that are highly visible. Highly visible and likely to evoke accidental clicks are, unfortunately, highly correlated.
* Most advertisers who are paying per click are very performance driven. They look at ROI, which means they look at conversion rates. When you charge an accidental click, it's almost certainly not going to convert, so in the end you look worse and they pay less for your clicks.
* One unfortunate bit of truth: advertisers do pay too much attention to clicks and CTR (click through rate). Even advertisers doing brand awareness campaigns, which are not looking for immediate response from their audience, tend to look at CTR.
* Yahoo has actually tried hard to establish other metrics that they should look at, like "Bounce Rate", which attempts to factor in whether visitors immediately exit after clicking. They use those metrics internally for optimizing ad performance, so accidental clicks are likely to discourage showing an ad more than encourage it.
I think there's a very non-trivial chance the person who wrote this article knows much more about advertising than you think: http://www.hilarymason.com/about/
To be fair, it's not clear that Hilary Mason is specifically knowledgeable about on-line advertising. She is undoubtedly very knowledgeable about computer science, big data, and a number of fields in general, but it's possible her major interaction with the specifics of on-line advertising are as an advertisee, either directly or by proxy.
To be clear, I'm not knowledgeable about online advertising. I do understand that the incentives of the advertisers and the users are often opposed, but I was highlighting this as an example of poor product design likely due to optimizing click-throughs, not product experience. :)
I think the notion that poor product design can evolve from abuse of data is an interesting discussion, but unfortunately a) it probably isn't poor product design (what function would you put in the areas of the screen most likely to suffer a misclick?) and b) not at all likely due to optimizing click-throughs. Ironically, the misclick problem is a well known issue in online advertising, and the best way to mitigate it is to use the data to dampen the noise.
There are obviously a number of ad revenue driven pressures on Yahoo Mail's design. The display advertising market is very different from the search engine market. It is only tangentially about the clicks. The biggest pressures are:
"Premium Locations": these aren't necessarily the places that get high click throughs, but rather the locations which are prominently displayed and noticed by the users, which causes them to stand out in a world awash in banner ads. This competes with good UI design, which wants to put important functions in these exact same locations.
"CPM": just getting way higher impression counts. This translates to filling the screen with as many ads as possible, stealing screen real estate and really disrupting eye flow.
"CPA": Lots of ad networks/DSP's try to arbitrage between CPM and CPA. Misclicks don't generally help much with that beyond hindering optimization efforts to use clicks as signals. Mostly what this does to design is encourage "rest points" in applications where users are likely to actually take the time to go all the way through to a conversion (think of it like TV commercial breaks in sports).
"Uniques/F-Cap": You want to raise the bar on the minimum # of impressions seen by every visitor, because those early impressions are far more likely to be people who haven't seen the ad before. Advertisers pay a lot more (10x more is not unheard of) for that. So, there is pressure to spam the visitor with as many impressions as possible before they are likely to leave (which makes it hard to serve ads in a way that is proportional to the value the user derives from the experience).
"View based conversion": The underbelly of the adverting network. In theory this is a good thing. In practice, it creates an incentive to shove ad locations in to the page that the visitor never even notices (ironically the opposite of the problem the article contends).
Some people have apparently gotten the impression I was questioning your credentials, your intelligence, or somehow attacking you personally. At least in terms of intent it couldn't be farther from the truth! While I think your naïveté about online advertising undermines your article, all it might imply about you personally is your curiosity about online advertising is low as compared to other pursuits. If anything, I'd say that is a compliment. If I conveyed or you perceived anything other than that, I certainly apologize.
Perhaps someone felt that it inappropriate to call the author out by name, and didn't feel that referring to them by a completely appropriate gender pronoun was in any way offensive... because, you know, being a woman isn't some kind of insult.
If it had read, "He's a data geek..." it would have only have been offensive because they got the gender wrong.
"... Perhaps someone felt that it inappropriate to call the author out by name, and didn't feel that referring to them by a completely appropriate gender pronoun was in any way offensive. ..."
I understand the line of reasoning, however the tone of the comments about the article are negative and not up to standard. [1] It is not a question of gender but how to discuss, acknowledge & discuss an authors work in a civil manner.
I agree the tone of many of the comments are not up to standard. I think both mine, and and the one you chose to go after were. Neither of us questioned her professional skills, talents, intelligence, awesomeness etc., just her domain expertise (in a domain which I'd think no one would expect expertise), which seems eminently reasonable given that the only indicators were a lack of said expertise. It seems even more reasonable because she acknowledged that she does not.
You are criticizing the people who are behaving in a civil manner for the behaviour of those behaving uncivilly. Try to focus on the jerks, there's an off chance they might get it.
"... You are criticizing the people who are behaving in a civil manner for the behaviour of those behaving uncivilly. ... Try to focus on the jerks ..."
Yes & no. I'm don't think I'm being that critical, 'why don't you refer to someone by name.' I take your point though. As for focusing on "Jerks", I ignore them. I'd rather point to measured responses like your own illustrating to "Jerks" how to respond.
While I agree that being a data scientist increases the odds one might know a thing or two about online advertising, it's far from a strong indicator, so I'm not sure what your basis for suggesting a "non-trivial chance".
Ms. Mason has confirmed her lack of domain expertise, so I guess that settles it.
In the process of attempting to highlight an important lesson about applying data analysis, she has highlighted another important and related lesson: regardless of the level of one's data analysis expertise, without domain expertise to contextualize the data, it is terribly difficult to do any kind of useful analysis.
I didn't say she was an idiot. Honestly, the way the ad business works sometimes, it makes more sense to idiots than to smart people. ;-)
A lot of people with her background come to the online ad business. While brilliant, they find they have a lot to learn and a lot of assumptions that they need to throw out the window. Fortunately, the brilliant ones have no problems learning so they come up to speed quickly, but this doesn't help them to understand things prior to digging in a bit.
Really? Because nothing in that bio points to that. Actually the whole thing is unbelievably vague and opaque.
And these kind of cutesy oneliner descriptions of one's skills make me cringe: "Simply: I make beautiful things with data".
OK, she works as a "chief scientist" (that's not even a job, for us, old time folks, but anyway) at bitly (a URL shortening service, i.e as far away from a real business as you can get, that aspires to be a "bookmarking service" also.
> a URL shortening service, i.e as far away from a real business as you can get,
Please define 'real business'. What does a company need to have to be a 'real' business? Users? Revenue? Profits? A product that at least some people are willing to pay money for?
I agree that bit.ly is a very simple concept, but I take issue with the disdain in your post - even URL shortening stops being straightforward once you expand from n = 1 to n = 10,000,000. And bit.ly does a lot more than 'just' that.
This reminds me of McCain's 'real America'. You may not like {NoVa, these kinds of startups} and you may think that they don't capture the 'true' spirit of {Virginia, business}, but at the end of the day, their {votes, dollars} count just as much as 'real' {America, business}.
You may not think Hilary's qualified to talk about the details of Yahoo's particular advertising model, but don't extend that into what's bordering on an ad-hominem attack against a very legitimate startup, as well as a key figure at said startup.
>Please define 'real business'. What does a company need to have to be a 'real' business? Users? Revenue? Profits? A product that at least some people are willing to pay money for?
Revenue and eventually profits. Even if "url shortening" is a "real business", it still is in the very far outskirts of "businesness". A very marginal value adding service, that exists solely because of some very marginal deficiencies of other services. Even Twitter has trouble monetizing, a "url shortening service" 100 times more so.
>even URL shortening stops being straightforward once you expand from n = 1 to n = 10,000,000.
I fail to see how. Even a simple setup can handle 10,000,000 shortened urls with aplomb. And url shortening is the most sharding friendly use case you can get, ie linear scaling is trivial in a url shortening service.
>You may not think Hilary's qualified to talk about the details of Yahoo's particular advertising model, but don't extend that into what's bordering on an ad-hominem attack against a very legitimate startup, as well as a key figure at said startup.
Well, she may or may not be qualified. I just pointed out that the page the parent pointed to as proof of her competence on the matter doesn't _prove_ her qualifications at all.
[Disclaimer: I have decades of first-hand knowledge of Hilary's awesomeness, going back to when we were CS students together in college. So yeah, I'm defending my friend.]
I'd like to ask you to think twice before publicly questioning someone's credentials like this. Whatever your intentions, picking on someone's CV just because of a blog post you disagree with is not only rude, but it sends a message -- particularly to women in tech -- that if they speak publicly, if they offer up their opinion, they will be attacked not about the content of their point but about their competence to speak at all. I believe this kind of attack has real consequences on our field, and I would urge everyone to show everyone the respect they'd want for themselves.
>it sends a message -- particularly to women in tech -- that if they speak publicly, if they offer up their opinion, they will be attacked not about the content of their point but about their competence to speak at all.
Oh, don't worry about that. I'm an equal opportunity insulter! I didn't even care that she is a woman or not while making my point. Nor do I think that women in tech need special treatment. We all get what we all get. Do you see exchanges between males being any more courteous?
Notice also how I avoided to speak about their opinion on Yahoo at all -- they might or might no be 100% correct.
I only responded to the statement by some parent poster, that her CV "proves" that she is especially knowledgable in the ads field.
At a small company with specific goals, there's no implication of doing "'science' in general". I can buy that the idea of being chief scientist at, say, IBM or Google or Microsoft is a little fluffy, but if you're a startup focusing on one goal that is a little beyond what the scientific community knows how to do in theory, it's entirely reasonable to have a position in charge of guiding and directing (and performing) research to that goal and keeping up with the state of the field.
It's largely equivalent to a PI role of a university research project, except that there are things going on in the company other than pure research and so there are chief officers of other things too. Nobody thinks that "primary investigator" is a fluffy title on the grounds that people don't do investigation in general; it's clear they're investigating specific things.
You say "Nobody does 'science' in general", but you are wrong. Many people work at organizations that have a broad range of scientific pursuits in a broad range of fields of study. The Chief Scientist's job is "ensuring that [the organization's] research programs are widely regarded as scientifically and technologically well founded and are appropriate for their intended applications".
Dude, Amgen and a ton of other biotech/pharma firms have "Chief Scientists" or "Chief Scientific Officers" on staff. Sure, they work in specific fields or specific departments, but that is absolutely a valid and commonly used title.
Edward Tufte makes the exact same argument and has a book entitled 'Beautiful Data,' and NO ONE says that about him. Also, while I applaud your use of quotes as rhetorical device to discredit Hilary Mason and her job title, I respond better to well-reasoned arguments.
May I encourage you to watch this video: http://www.youtube.com/watch?v=qL3DWcBsPU4
Now imagine her saying, "Simply: I make beautiful things with data." Does that still read cutesy to you?
I might also suggest perusing her GitHub code, but I wouldn't want to alarm you, what with all this newfangled technology and rapid motion.
> * It turns out what most impacts ad effectiveness online
> is whether people actually see the ad. ... Highly visible
> and likely to evoke accidental clicks are, unfortunately,
> highly correlated.
A good summary of the OP's point, I think.
I'm not sure what your end point here is, as it's all over the map, but you seems to simultaneously be arguing against the author's point and for it as well.
> I'm not sure what your end point here is, as it's all over the map, but you seems to simultaneously be arguing against the author's point and for it as well.
Sometimes the truth doesn't entirely support or undermine someone's argument. It's one of the sure signs that it is the truth. ;-)
I wasn't trying to make a specific point, but rather to inform everyone, since the article didn't.
I don't mean to be coy, so I'll share my opinion:
The article presents this as a bug, which implies it is clearly wrong. It isn't. Misclicks are always going to create undesired outcomes. If you didn't have an ad there, the misclick would trigger some other action. Showing an ad landing page is probably one of the least harmful things you could reasonably do with a click in that location, so I question the notion that this leads to a "terrible experience". It might not be a great experience, but unless Yahoo Mail is expected to fix mice and operating systems, it's probably one of the better experiences they could provide in response to a misclick.
Ms. Mason seems to believe that the false clicks are driving Yahoo to think that they are getting superior ad performance. That isn't the case, as the data itself does indeed provide some clues as to the relative likelihood that an ad received a misclick.
The notion that this is evidence of "the data only takes us so far, and creativity and clear thinking are always required to find the best solutions" isn't well supported by this problem. In reality, clear thinking about the data itself has indeed lead to about as good a solution as Yahoo Mail could provide.
I'm not sure if I understand your criticism. The article seemed to be commenting about the placement of ads (which she believes to be optimized by analytics), not about the economics of ads or how they are processed behind-the-scenes. I come from a data understanding background, and the cool thing I liked about this article was the following statement:
"...experiences like this are great reminders that data only takes us so far, and creativity and clear thinking are always required to find the best solutions..."
This might be very basic knowledge for everyone, but many researchers in machine learning & data mining community do not think like this. They believe that data tells the entire story.
I come from a data understanding background, and the cool
thing I liked about this article was the following statement:
"...experiences like this are great reminders that data only
takes us so far, and creativity and clear thinking are
always required to find the best solutions..."
I think in principle the statement was a good one, but unfortunately the case study doesn't fit it. The location of the ad isn't influenced by optimizing click through rates, as she presumed. It also, isn't really bad design, because there isn't a much more harmless thing to do with a misclick than to show an ad. Most importantly though: the way to mitigate the damage from misclicks actually is to use the data.
In the screenshot the ad blends into the rest of the UI instead of being clear that its an ad...And its prominent placement in the UI makes it very clickable. Following your logic shouldn't it remain in the same place but be more clear that its an ad so that people are not confused that its part of the product? This would have the result of being seen but not accidentally clicked.
First: it may not be quite so apparent from the screen shot, but the ad is framed to make it distinct form the rest of the app. Going further might well do more harm than good, as it could disrupt sight lines and make the UI garish regardless of ad content. The ad itself also draws distinction with the surrounding widgets by employing a different font, a different icon style, a different background, and a different icon size.
As the article describes, people aren't misclicking because they've looked closely at the ad and mistaken it for an application function. Misclicks are predominantly people who really aren't sure what they are clicking on, don't know they are clicking, and/or didn't intend to click at all.
Either way, in practice Yahoo's optimization efforts if anything create incentive to minimize misclicks, not the other way around.
"Most Yahoo Mail ads generate revenue based on impressions, not clicks. So Yahoo isn't directly making more money by grabbing these "random" clicks"
perhaps not, but surely if I was to buy such an ad and pay per impression, I'd probably like to know what the impression/click ratio was. no doubt Yahoo advertises a low one to potential buyers...
As someone who has been in the belly of the beast, I'm going to try to summarize the various ways that this article is getting it wrong.
* Most Yahoo Mail ads generate revenue based on impressions, not clicks. So Yahoo isn't directly making more money by grabbing these "random" clicks.
* Where clicks do matter, major ad platforms including Yahoo throw away a LOT of clicks as fraud, and accidental clicks tend to disproportionately get thrown out along with them.
* In general, random clicks are considered a real PITA for major ad networks, as they confuse the heck out of ad optimization. While small players do tend to soak up that revenue, the big players really, really hate the phenomenon because it makes them far less efficient.
* It turns out what most impacts ad effectiveness online is whether people actually see the ad. An incredibly number of ads are just never seen by the audience. As a consequence, a good publisher will try to find locations for their ads that are highly visible. Highly visible and likely to evoke accidental clicks are, unfortunately, highly correlated.
* Most advertisers who are paying per click are very performance driven. They look at ROI, which means they look at conversion rates. When you charge an accidental click, it's almost certainly not going to convert, so in the end you look worse and they pay less for your clicks.
* One unfortunate bit of truth: advertisers do pay too much attention to clicks and CTR (click through rate). Even advertisers doing brand awareness campaigns, which are not looking for immediate response from their audience, tend to look at CTR.
* Yahoo has actually tried hard to establish other metrics that they should look at, like "Bounce Rate", which attempts to factor in whether visitors immediately exit after clicking. They use those metrics internally for optimizing ad performance, so accidental clicks are likely to discourage showing an ad more than encourage it.