Besides the novelty factor (which others have mentioned), his 'paint' version is also much simpler, and therefor much easier to understand at a quick glance.
- Large text
- Not very much text
- A single image
- A single visual flow (top to bottom)
- A concise color palette (greens and black)
Compare to the 'standard' ad, where:
- Text is too small
- Too much text
- Too many images competing for attention
- Muddled visual flow
This is what I wanted to say. I kinda mentally block ads* and I saw the Paint drawing in the article before the "actual" ad, even if it is in the reverse order in the page flow.
* A few years ago I missed a question in an exam that was framed and underlined. The goal was to emphase the question because it was important. When I learned that I missed the question (while discussing the exam with other student), I took a new look at the question and my only guess can be that it looked too much like a Google Ad at that time and despite the fact that it was on paper I unconsciously ignored it… This is how stupid I can be.
I might have skipped that question too, but I would guess it's because frames and underline typically denote instructions on a test rather than an actual question.
Definitely agree the fancy ad is filtered out subconsciously as it's an obvious ad.
I disagree that the simple ad looks like part of the content (unless the rest of the page is kindergarten kid's drawings). It's unique and an eye catcher. Until everyone starts doing it and then it'll become filtered just like the "fancy ad".
Even more importantly, a high CTR with lower conversion rate would drive up your costs while lowering your sales.
In this case the first ad is very clearly for a game with a very clear target market. If I am in to racing games, I may click it and I may buy it.
The second ad could appeal to people strickly on novelty which means you may get a lot of customers who do not care about the product.
While the results of this post are very interesting and the conclusion to test everything is still good, we should look at the conversion rate to see which ad is actually more effective.
Interesting point. Obviously makes a big difference depending on whether you're the advertiser (selling products) or the publisher (getting paid for clicks)
CTRs aren't irrelevant, since they're a concrete proxy for conversions that ad networks can relay back to publishers quickly, but nobody actually pays by clicks anymore. That was common fifteen years ago, but CTRs plummeted circa 2002 and never recovered. Display ad revenue is now invariably calculated from impressions.
I work in Facebook app monetization and mobile ads, it is almost all CPC and CPI. The ad ecosystem is diverse enough that different pricing models apply in different situations.
I think another big factor is probably a white background vs. a black background. Bill Watterson said he avoided making strips with dark backgrounds too often because the human eye is drawn away from dark to light.
I would like to see an A/B test of the same basic artwork on a black background vs. the white one.
I do as well, but we're outliers. In an A/B test, the version which emphasizes free usually wins. I've even seen it beat an offer of getting paid to do something.
Good lesson on the value of in-market testing. That said, would love to see some theory and analysis about why the MS Paint ad outperformed the standard ad. My hunch is that the first ad -- while it obviously looks a lot more professional -- looks like every other banner ad on the internet. It reeks of ad-ness, and it may set off some psychological barrier to receptivity amongst viewers precisely because their brains have been trained to filter out ads. (Banner blindness, as one of the other posters has pointed out).
Conversely, the MS Paint ad is, if nothing else, novel. It looks pretty different from most display ads out there. It catches the brain's attention, rather than being caught in the brain's passive ad-filtering heuristics. This may be, if nothing else, a story about attention and awareness.
I think you're absolutely right. I would wager that if every ad on the internet was changed to a five-minute MS Paint scrawl, click-through rates would rapidly go back to where they were originally, if not drop even further.
The novelness lies in the lack of polish. People who work in advertising like ads, and well-polished ads are more pleasing to their eyes.
To the rest of us, advertisements are used as something we don't want to see, but are willing to see, in order to see something we DO want to see. Many of us can pick out the disparity between the polished, targeted ads and the less-focused content it intersperses.
The MS Paint ad was much more readable, IMO. Too many adds try to cram too much into their restricted screen real estate, with the result that they're much less clear.
Factor in that most ads are competing with others alongside (and they all look the same); also as others have mentioned, we've adapted to filter out 'obvious' ads.
Actually, it's more likely because 80+% of Google users couldn't (and probably still can't) tell the difference between the advertisements and the results:
This is absolutely true. The background colour is slightly different, but on my laptop screen that's hardly visible at all, unless I tilt the screen at an extreme angle[1].
I have to say I kind of feel cheated by that trickery, every time, and it is one of the reasons (though not the most important one) that I use Google much less recently (in favour of DDG).
[1] no I'm not particularly happy about this screen, the rest of the machine is great but if I had known it would be this bad I might have picked a different model (I got an Asus EEE 1215B)
I've seen pages where the Text Ads have the same Style as the overall page. The only thing that sets the Ad apart is a small grey text 'Ads' or the like.
I've may have seen far more of such pages. But obviously i can't tell since they're near-impossible to spot the Ads.
The trend is keeping things "by the book" from search engines point of view, And manipulate humans in subtle ways
by confusing contents and ads.
I think that my comment still applies. We've gone from a smaller number of people with a high concentration of those that are 'tech literate,' to a larger number of people with a smaller concentration of the 'tech literate.'
That's probably some of it. I could be wrong but I think the ratios have gone up more than that. Anyway, I don't think the "techies don't click ads" effect is really as strong as a lot of people assume.
My guess is that (a) Google has optimized the ads for more clicks (b) advertisers have gotten beter at attracting clicks & (c) People are more aware of when to click on ads. If you're looking for a babysitter, you are more likely to have one booked ten minutes from now if you click an ad than an organic result. People have, consciously or unconsciously become aware of that and they click the ads when they want that kind of content.
> I don't think the "techies don't click ads" effect is really as strong as a lot of people assume.
I kind of doubt that myself and would like to see some numbers before coming to that conclusion. Especially if just talking about CTR instead of conversion rate. For the latter I'm willing to believe that if the techie clicks the ad, they're more likely to buy. Though that is a guess I'd also like to see supported with numbers :)
It might also be very different in the USA. If I see an ad with my search results, a lot of the time it's a US company and even if I would want to use a service on the other side of the ocean, I can only pay with either my creditcard or PayPal. I won't use PayPal and my creditcard is tucked away in some drawer somewhere. Actually, for any non-Dutch company (say, German), a creditcard or PayPal is usually the only option. On a Dutch site I can pay with iDeal via my bank card (zero transaction costs, authentication via text message on my banking account's site).
It might also have something to do with giving the impression that the game is going to be something novel, quirky or even bizarre, as opposed to another polished but bland EA title.
Interesting, but the author is missing a critical point: the conversion after clicks. When I see the first ad, I clearly understand immediately that it's for a car game. I'm not likely to click, but if I click, I know what I want, and I'm very likely to actually download/install/play the game.
When I see the second ad, I wonder what it is, so yes, I'm more likely to click on it. But I'm not interested in car games. So I'm also much more likely to press the back button as soon as I understand what I'm being sold.
How does the math play out in the end and which ad is betetr? There's no way to tell from that data.
Hey alain94040, I didn't have time to test for CVR :( It would mean split testing demogrpahics/bids and possibly even landers. The point that everyone should take home is just to test every idea that comes to mind :)
We find with domains with landing pages you can have thousands of visits per day that result in no revenue if the reason for the visit (domain typed in on browser bar) is not related to what the person is actually looking for.
Both ads advertise Free, and it's clear that it's a car game in both ads... Also you need to consider the fact that higher CTR means higher relevancy score and potentially lower costs if you're paying by the click.
I think the comments in this thread about ctr vs actual conversions are a bit silly.
Pof's target audience here is "people who like video games" and they're trying to get people to play a free online game. They don't really have anything to teach people in verticals where the consumer is making a measurable commitment. You do not need to be very persuasive to "convert" with that crowd.
Almost all marketing advice more specific than "the customer is always right" and "sex sells" etc. is dependent on the product and the audience.
What this is is a great ad. It's really funny (The speed lines especially. This car does not look very fast, but the speed lines show it's got a lot of heart), and it understands its audience. I think it has broader application than many are giving it credit for, it probably would work with any sincere product (i.e. selling something other than "one simple rule") targeted at younger people that does not solve a "serious" problem. Once you are proposing that people spend more than $40-$50 you are pretty smarmy if you are trying to push them into an impulse purchase
No one should decide on a health insurer based on a MS Paint ad, no matter how hip, nostalgic, and casual they are. On the other hand, you can make a lot of money selling funny ads to people and I think that's what's going on here.
Amazing list. I would perhaps add to this the "market sophistication level". In this particular case (saturated market and lots of high quality competition), ethos is highly important.
The ad is successful in empathizing with customers by display a drawing that follows the design of top-of-mind ad very current memes. This works because of the market sophistication level (i.e. potential customers knows how this particular video game works).
Users haven't seen an ad like this one a thousand times before. Some might not even think this is an ad. Some will be curious enough to actually click.
The MS Paint banner ad is not inherently more effective. If every banner ad on the internet was hand-drawn in MS Paint in 5 minutes, the joke would quickly grow old, and the CTR would vanish.
He also looked only at CTR and not CPA. The sort of people who click (due to the novelty) may not be the sort of people he can convert. His overall CPA may well be lower with the other ad.
Yes. It's your basic revenue vs. profit error reworked. CTR is a poor measure of effectiveness - a lower ctr can be better on ppc if you're converting better.
I agree on the relevance of the novelty factor. It also came to my mind that this kind of childish MS Paint graphics are becoming somehow pop-culture, a meta-internet-meme if you want. Yesterday I spent half an hour on Christopher Poole's http://canv.as/ (just because I ain't brave enough to go to 4chan) and noticed a lot of these graphics; after some time lurking, I started considering them kinda cool. It's like... well, a brand.
The "test everything" mantra sounds good, but in practice, you generally have only so much data you can afford (in impressions per day, or whatever), and when your CTRs are often 0.1% or lower, you need quite a lot of data to get narrow confidence intervals around your CTRs. Using the basic binomial model, if you have two test conditions, one of which actually does 20-25% better than the other, (say, 0.11% versus 0.09%), your confidence intervals will keep overlapping until you have OOM 1M impressions. This is all just to say that running a whole lot of tests can quickly become expensive an impractical.
While testing some radical, weird treatments can give you valuable perspective, or shed light on the assumptions you've been making, testing every idea is rarely feasible. I would not, for instance, guess that that the author should test different versions of the second ad with colors or number of exclamation points changed.
As someone who has spent $100,000 on advertising over the last few years, at least $10,000 of which was on the Plenty of Fish platform, you're very wrong.
You only need about 1,000-10,000 impressions to get an idea of how a creative performs. Often less. As you get more and more used to each particular advertising platform, you also get a feel for how an ad is performing.
In my business, a difference of 0.02% CTR could mean the difference between earning 30% ROI and 50% ROI - the words "test everything!" mean everything to me and my results.
Yes, test everything. But what exactly do you mean by "get a feel"?
Suppose your CTR is known to be either 0.09% or 0.11%, you've had 10,000 impressions, and you've got 11 clickthroughs. (This is of course the most likely number if your CTR is actually 0.11%.) The likelihood ratio between the two possibilities is about 0.81. So if you thought those two possibilities were equally probable before, you should now think it's about 55% likely that the CTR is 0.11% and about 45% likely that it's 0.09%.
So if by "get an idea" you mean something much stronger than 55%:45% then I fear you may be fooling yourself, no matter how much you've spent on advertising over the last few years. (Whether that means you should reconsider "test everything" depends on the costs of testing -- the actual cost of doing it, and the cost in running something other than the currently-believed-best version.)
And with 1000 impressions? Forget it. You expect to see 0.9 clickthroughs on average with a 0.09% CTR and 1.1 on average with a 0.11% CTR. You can probably get some extra information from (e.g.) when that expected single clickthrough happens, but it's not going to take you near to that 55:45 ratio. (I might believe 52:48.)
First of all I don't care how much you've spent on advertising, or how much experience you have. Human brains are hardwired to see patterns that don't really exist. If you haven't done the math then I guarantee that your intuition for what matters is way wrong.
However going the other way, people take standards from the science world into A/B testing that are not appropriate. If you're testing a ton of ideas, getting the right answer 3x out of 4, and not going far wrong most of the rest of the time is a pretty good result. It is certainly a lot better than concluding that it is too hard to test those ideas at all.
But if you have a specific idea you want tested, or if you want all ideas tested to a certain confidence, then you really, really need to either do the math, or to get someone to do the math for you. Because the human brain is a pattern finding machine, and you want real answers, not made up stuff.
I think part of the point abeppu is making is whether or not the two the statistics for each advertisment type are different. You see a difference of a 0.02% CTR but is that significant?
An analogy (from an interesting post I can't find) is tossing a coin for two samples sets of a 1000 times. In one sample I wear a read jumper and in another I wear a green jumper. I find that the green jumper gives me a 0.02% improvement in producing heads. Therefore I will always wear my green jumper in future when I toss a coin.
Obviously however this is just random error. A statistical analysis of CTR will tell you if the difference between the two advertisments you are observing is significant. The larger the samples size, as abeppu wrote, the greater the greater confidence in your results.
I agree with your point, however I would like to highlight your use of confidence intervals:
"If two statistics have non-overlapping confidence intervals, they are necessarily significantly different but if they have overlapping confidence intervals, it is not necessarily true that they are not significantly different."
In particular, it doesn't scale well at all with the number of choices (the familiar curse of dimensionality). You can reasonably test two variants, but if your design could plausibly vary along, say, 10 axes (not uncommon), you're going to have trouble collecting sufficient data to cover the whole 10-dimensional theoretical design space. So data-driven design can usually only be applied to a small part of the design space, typically testing a small number of alternatives.
The problem is, that this only works because it's so completely different from all the other ads and because of that reason escapes some of the banner blindness.
We've had similar results when we modified our logo and added mistakes. For example a rectangle Google AdSense Banner with a mirrored version of our Logo or just some crazy saturation affect applied on top of it actually got excellent CTRs. Even though it was kind of unnerving to have all my friends tell me about the mistakes that were in our ads...
Actually advertisers have known this since a long time. Ugly wins more attention. More attention = higher conversion.
The first job of ads is winning attention. You have to fight against all the clutter and stand out. You can do so by a variety of tactics. Use human faces. Use cute looking models. Use ugly fonts or clipart. Use mouse pointers. Use fluorescent colors. Use dashed coupon type borders.
You need to know however that while your conversion rates will increase, the number of complaints you receive will increase too.
The most over-the-top-ugly banner ad I've seen was a "Would you like to go back to school on a grant?" ad targeting women. It had a cartoon woman in a superhero costume with a blinking siren pasted (seemingly) randomly on her head.
No, but my mom probably would have, and it got my attention enough to where I can describe it precisely _and_ still remember what it was advertising, though the image and the product have nothing in common. Woah.
This is actually well known among savvy internet marketers. And it's why those tips for belly/whitening/wrinkles "invented by a local mom" always look far from professional.
It works because:
- It stands out from the content
- It defeats ad-blindness
- It's not expected, so it makes you curious
If this style becomes the norm, users will learn to mentally filter ads made in Paint as well.
With the # of ads I've seen with that tag, I'd imagine the "local mom" to be some sort of genius, making $5000/week, getting dermatologists to hate her b/c of some wrinkle miracle while losing 800 lbs in 2 weeks.
> With the # of ads I've seen with that tag, I'd imagine the "local mom" to be some sort of genius, making $5000/week, getting dermatologists to hate her b/c of some wrinkle miracle while losing 800 lbs in 2 weeks.
800 pounds indeed. It's because of this incredible weight-loss that "local moms" have an extremely short half-life and are in fact rarely observed outside laboratory conditions.
Actually, this is a scam not just because it doesn't work. It's a scam because they make it sound like it's something free or extremely cheap, when in reality it often initiates a costly recurring billing (rebill) agreement through careful wording in the fine print. Gullible people often end up paying hundreds of dollars in the end.
FIRST: tylerrooney, thanks for posting my blog on Hacker News! Much appreciated. Judging from the comments, we've got some super knowledgeable people here.
So to clarify, this was simply a CTR case study, I know there's another side of the coin for CVR but that would have taken more time and funds than I would have been willing to allocate :(
But hey, open invitation the community here: If you want to submit a 310x110 ad for the purpose of testing against the same demographic that I did, feel free to email it to me: ben@pof.com. I'll run it for a few days and I'll let you know how well your ad did :) And to make it worth your while, highest CTR ad (from Hacker News members ONLY) gets $100 credit to advertise on https://ads.pof.com. AND you can use your affiliate links so if you make some coin, it's yours to keep (Put it towards your Diablo 3 pre-purchase, yeah? lol) Just grab the direct link for Need For Speed World from your favorite network and send it over. And hell, if your ad beats mine, I'll post it on the POF blog, with your permission of course.
End date for this little challenge.... April 30th?
This is hilarious, but it doesn't take into account one of the major factors of online advertising, and that is branding, or brand awareness. While the official EA ad may not get as many clickthroughs, what it will get is subconcious eyeballs, and given enough impressions of the same ad in various formats, same EA logo, car, game title and branding, you don't need people to click on the ad to start to recognize that there is a new Need for Speed game available. It's similar to flooding the airwaves with a particular TV ad. The goal is awareness, which over time, can lead to a purchase, whether the person decides at some point to purchase online or offline. Maybe they're at GameStop and EA has placed an in-store display with the same EA Need for Speed branding. The person may have forgotten about the game, but walking in to this GameStop, they're memory is refreshed of it by being previously exposed to it through digital or television.
So in short, what's more important? Immediate click-through satisfaction or building real brand recognition that can show greater returns over time, mostly in ways not calculable?
I work with many MBA colleagues and whenever they encounter an ugly, but effective (ie. converting) ad, they can never drop their egos to accept that such an ad can work. They just feel they cannot "stoop that low" to adopt these techniques.
They would often blindly push for "simplicity", "sparse text", "nice picture", and when these ads go out to market, they get absolutely crushed.
One of the tragedies of a big wealthy company is that marketers can continually go out with these crappy creatives that don't sell and there really is no big consequence. It is often written off as a "learning opportunity".
Whereas if you look at the ads of people whose lifeblood depends on selling their product, they may not be the most attractive ads, but the ads that persist over time tend to be effective (ie. they sell). These guys need to eat, and they can only afford to make stupid ineffective ads for so long before they starve! So there are definitely some practical lessons that can be learned from them. They often knowingly or unknowingly follow the principles of advertising legend David Ogilvy.
Their ads tend to hit on direct marketing best practices:
- headline states in plain language what the product does (ie. no MBA jargon/buzz words)
- headline also hooks the reader to read a bit more
- it is clear who the product is for
- copy combats any objections in reader's mind
- no distractions that divert reader from clicking the "Join Now" button
- contains customer testimonials reader can relate to (ie. social proof)
- gives reader enough information on the page to make a decision (ie. none of this sparse text BS if it doesn't make sense)
Is it just me or did anyone else find the first ad terrible and the second one quite good? I think I have been in the game too long.
One of my favorite ads of all time was the stick drawn fat girl with the secret to losing weight. I am sure the person behind that ad has a serious bank account.
I love this kind of social experiment. There are a few theories as to why this resulted in a higher CTR. The obvious ones to me are:
1) "Look at the pretty picture!!"
2) "What the hell? EA is allowing this ad? Did they make it? I gotta see where this goes!"
3) Alternate to "Banner blindness", as @jonnathanson pointed out.
In 1 & 2, I feel like the higher CTR wouldn't matter because people are acting on curiosity of the implementation rather than the product. Once the outcome was revealed, and the banner is seen as no more than a trick, I'd be willing to bet that the orders or pre-orders of the game (in this case, playing for free) stayed roughly the same as if using the other banner. Just a hunch.
If you guys think this ad is on to something, you really should see the ads put up in the SomethingAwful Forums (general subforum: http://forums.somethingawful.com/forumdisplay.php?forumid=1 ). The interesting thing is that many (I'd estimate more than half) of the ads are put up by forum members themselves, making for a range of very informal, hilarious, parodic, often obscene ads that play to every stereotype ironically and knowingly, sometimes made shittily with MSPaint.
And where do those ads lead? Usually to a forum thread where members are playing/raiding an MMO together, or discussing a topic of great interest, or selling a service like painting portraits, web hosting, or resume editing (and in one or two cases, to a discussion of a particularly zeal-inspiring anime series).
This is definitely interesting. I know I'd be more inclined to click something that looks like that ridiculous paint drawing. But this bit bothered me:
Results? 0.049% CTR vs. 0.137 CTR
I hope I'm not the only one confused by this, but the lack of a percent symbol on the 0.137 means he went from 0.049% to 13%, an improvement of almost 300x.
Is this a typo? Did he mean to say he increased his CTR to 0.137% or did he actually increase his CTR by 300x?
Edit: Why is this being downvoted? It's a legitimate question.
You might also want to consider using promille (per 1000) or something similar, when dealing with such tiny percentages. (I don't mean they're tiny for a CTR figure, just that small numbers like that are harder to get a gist of than "1.37 clicks for each 1000 views".
I wonder if the inspiration for this test was directly taken from Max Teitelbaum's interview @ mixergy?
Search for 'paint' and you'll find the relevant passage.
"Andrew: Give me an example. I’d love to.
When we started media buying, we saw stock raising.
We saw what all the brands were doing. We needed
something really flashy, something really clean.
When I made something in Paint, as a joke and sent it over
to the guys, we just threw it up as a test, and it
quadrupled the conversion rate of anything else.
After that, I think we made every banner in Paint."
As joblessjunkie mentioned, novelty is a huge draw. I've seen ads on Facebook, Yahoo and MSM sites (probably delivered by ad networks) that feature eye-catching faces with some ridiculous or unusual element (long beards, tattoos, elongated teeth etc.), but are advertising something that's pretty mundane (loans, online education, etc.). They are much more noticeable than ads that use standard art or photographs.
Judging by the fact that the advertisers are still using weird face ads, the CTRs must be superior ... but the interesting questions would be around the type of customer or conversion rates.
I think it's critical to not lose track of buyer motives and end goals when testing ads.
If I were clicking on the first ad, I would do so with the intention of downloading Need for Speed. The second ad, I would click to answer the question, "Who made this crappy paint ad? It's pretty unique."
If you are paying per click, the first ad is going to perform better and will give more relevant traffic. If you are paying per impression and just care about traffic, the second wins.
"Every idea that you have is worth testing, no matter how crappy you think it is."
It took him all of 45 seconds to put that new test ad together. That is a very, very small time investment, thus making such a test worthwhile.
For instance, if one wanted to create some sort of live-action commercial with actors, lighting, equipment and CG, and test it as an ad, the investment of time and money would be so great that negative results would be disastrous.
The second ad is more "entertaining" in my opinion. Also "Ad Blinded-ness" might have caused the first ad to just go unnoticed among all the other ads. Being untypical helps on the web. But also, the point is that you cannot expect patterns to predict success always. The first ad was based on a pattern. Now, if we study the second ad for a pattern it might not work as well. Man, I love the web for this.
There's a perfectly good explanation for this. Eye tracking tests show that people ignore images and read around them (unless there's a human face in it). That's why text ads work better, in most circumstances anyway. The color of the second ad is lighter, and probably matches the background of the page better, making it less of a block and lowering the barrier to crossover and read the text.
I think this goes to show that we like low information density. Simple things are easier to absorb, high density things get blocked out naturally.
When we're presented with an large amount of information, our brain blocks it out rather than expending the energy to parse it. PTSD is an example of this which shows our tendencies to block overwhelming information.
Is no one worried about the image and association you create by the advertising campaign? I kind of feel like you dilute the overall brand value by going with low stupid tactics - it may create value in the short term, but does that translate into a stronger brand in the long run just because it has more people?
Cool, but I prefer the second one because I don't care about cars. To me the first one is very boring, the second I could click out of curiosity (just because it made me smile). Problem is: do you really want me to click that ad? Because I won't care about the game.
I remember seeing this ad (or one similar) in the Draw Something game a few days ago.
The context here helps immensely, I think. Draw Something is a game that's about crude drawing, showing an ad that's a crude drawing is the ultimate targeted ad.
I don't think we should discount the popularity of Draw Something. It's in the forefront of peoples minds and this associates it with that as well as being a fun game. I wouldn't be surprised if there is a strong correlation with the popularity of that game.
I have been running facebook ad campaigns for over a year now and consistently found that by hand writing something in paint on a relevant image, i get much better ctr than just the relevant image.
Not for social. http://techcrunch.com/2011/11/14/page-post-ctr/ quotes an average click through rate for Facebook ads of 0.05% (not the headline 0.14% which is about Facebook Pages CTR), so 0.049% is close to average and 0.137% is an excellent CTR for Facebook ads.
Agreed - particularly if you are the advertiser paying for a ton of "curiosity clicks" from people wondering where the ad goes.
It would have been interesting to see a measurable goal (downloads, newsletter signups, watch a 3 minute video... something measurable) and how the conversions were across the two ads.
The MS Paint ad still may have more total conversions but it would be interesting to see how the cost per conversion compares. If both cost per conversions are within budget, then by all means go full speed ahead with MS Paint.
i think this says more about testing than about ads.
update: i'm trying (and apparently failing) to imply that this article makes me question the idea of "test everything" rather than bolster the idea (the article's conclusion).
I think obfuscation is a factor. However, I think there are two other factors that should be considered: 1) the call to action is much clearer in the MS Paint ad "Play Free", and 2) the polished ad "looks" like an ad, whereas the MS Paint one doesn't, because "who in their right mind would make an ad that looked like this?"
On the second point, if everyone started making MS Paint ads then users over time will get used to the style as well, and stop clicking. People have an internal ad blocker that takes time to "learn" from existing patterns.
We once did an experiment with Google Adsense. Directly above the ad unit, if you used a large, clear heading "Sponsored Ads" and then compared the results with no heading, the clickthru rate differences were very significant. I don't have the exact number, but something like 3-5X in favor of no heading.
To improve CTR, you can either make a very compelling, contextual ad, OR trick your users into not knowing something is an ad. Tricking is easier and more effective in the short run, but will get you in trouble once your advertisers start looking at their conversion rates and ROI.
For example, the typical ad has 10% conversion but only 1500 clicks. You end up with 150 sign-up. The MS paint version has 5% conversion but 3 times more clicks, so you end up with 225 sign-up.
I bought a mini-site that monetizes with AdSense and made over 300% more than the previous owner by making the ad units ugly. The only reason I did it was because I didn't know how to customize the ad units to match the rest of the page like the previous owner had, so I just left them with the standard white background that made them look terrible. I kept meaning to figure out how to fix it but after about a month I noticed the huge spike in revenue and left it. The gains have stood for several years.
Conclusion: Ugly ads that catch your attention have a far superior CTR.
