The most obvious case of the Dunning–Kruger effect is when random HN commenters think that nobody in marketing is capable of using data-driven metrics. I don't think you appreciate how numerical modern marketing is -- marketers use conversion rates, A/B tests, attribution, and a lot of other techniques to actually calculate the ROI of advertising.
What you're describing may have been the case, in a number of companies, five years ago, but it's just not meaningful now-days. Companies spend marketing dollars because marketing dollars work.
Any other interpretation is ideology-driven wishful thinking. You might not like marketing, but it's silly for the conclusion to be "thus, the entire field is made of people who don't know what they are doing".
You might be underestimating the number of companies who absolutely don’t know what they are doing budget-wise yet they manage to stay in business, and threw a lot of dollars in ads/marketing without deeply analyzing its effectiveness, just because raising money at stellar valuations was free until a month ago.
Anecdotally, I happen to work for a 300 people saas company who just started drastically cutting expenses with the goal of reducing burn rate and delaying the need for another round of financing in this terrible environment: ads were one of the first few things to be dramatically cut (the second one was some super inefficient AWS spending).
The ads budget shrank from 6 figures a year to near 0 overnight, and it’s unlikely to return to its former level any time soon. Also, half of the marketing team was laid off. Having 30 people in marketing for such a small company was definitely way too much (just like engineering, sales, ...), and it anecdotally proves GP’s argument. All consequences of the fact that money was free.
If this is the case, you should be thinking about your own job. There are very few companies that can stay afloat without new customers. And even fewer SaaS companies as churn is amplified during a downturn.
Do you know what the CFO is telling their lieutenants about now? Cut all superfluous spending and reduce monthly expenses. That means your sales people are going to be getting calls from their clients asking for a fee reduction or a contract revision. Very few SaaS companies are critical or defensible.
I 100% agree, in fact a couple weeks ago I gave my notice despite me being insulated from the current layoffs. I had been wanting to do this for a while, and hearing the CEO giving projections of how much our revenues are going to shrink due to covid, compared to our crazy speculative company valuation, was the last nail in the coffin. They should have managed the company better and more frugally, so I vote with my feet.
This is my last week and I’ll be starting next week in a much more solid company with a much better compensation too, wish me luck :-)
I think you're underestimating how much big companies spend on online ads compared to small companies. Most big companies have big non-terrible marketing teams that know how to track the effectiveness of their ads. Most of the times, they have ROAS requirements. If the ads aren't making money, they get turned off.
> and threw a lot of dollars in ads/marketing without deeply analyzing its effectiveness, just because raising money at stellar valuations was free until a month ago.
And in turn, I think you're overestimating the impact of profit-limited capital dependant companies on the advertising landscape.
I was managing a 6 figure ad budget all by myself at an established but small non-tech company and had full E2E tracking flows for pretty much every £. "Burn" isn't a thing in almost all companies out there. You want to spend a dollar, you need to tell your boss exactly how it'll make you a buck fifty.
A young, growing company is going to throw whatever at the wall and see what sticks. That's just how to learn what works and what doesn't.
I wouldn't extrapolate that experience to large companies that have weathered dozens of downturns. I'm not saying that large companies all have a clue what they are doing vis-a-vis marketing, but there are a few giants who understand their only asset is their brand. Think companies like P&G or Nestle. Largely, the biggests differentiating factor between toothpastes or frozen pizzas is brand awareness and distribution deals. These are the companies that know exactly how much more money they will make if they change the color red to yellow on a box for a product sold in Georgia.
No it doesn't prove the claim that the marketing tram and budget won't regrow after the economy improves, nor the claim that marketing and advertising was being done wrong or unnecessary.
All we have to go on for that is your opinion about how the company was being run, which isn't backed by any sort of data.
A couple executives just plainly admitted how crazy inefficient our spending was, so I just can’t imagine how they could possibly go back to blow nearly a million a year in ads any time soon, or ever, when 90% of our revenues come from traditional enterprise deals that don’t have anything to do with ads, for the most part. They are all just worried that they will have to accept a down round in a year, and are trying to mitigate that.
By the way, this is true for engineering as well, where I work: now that we figured out how to remain productive while cutting 40% of our AWS costs (spot instances, killing idle instances more often, ...), will we ever go back to be inefficient and leaving idle AWS clusters up at night just because “money is not an issue”? Very unlikely. And the marketing department wasn’t being run any more efficiently than engineering, budgets were just fat because money was free for nearly a decade.
"I don't think you appreciate how numerical modern marketing is"
Maybe for the Fortune 500 / tech startup crowd, but I work for small / medium businesses and there is still a ton of ad dollars being spent with no attribution / tracking.
I will routinely setup clients with tracking tools, which then discourages them as they finally see what their digital ads actually generate in terms of revenue. Many stop their digital campaigns and increase their traditional tv/radio/print budgets or, fall back to running digital campaigns with no tracking.
The power and effectiveness of digital targeting is often overblown for small businesses.
Many of my small business owner clients hear anecdotes where a business was able to use targeting to create a veritable "money machine". The problem is these scnerios are fleetingly rare, complete bullshit or only apply to a specific niche.
The anecdote often glazes over the thousands of wasted dollars spent trying to perfect the model and a hundred other details that factored into the success.
The good thing for most is that digital advertising is no less effective than traditional advertising. Even if there's little to no targeting, or no tracking in place it's not any worse than tv/radio/print.
I can count on one hand the number of clients in my 20 years of web work that have created a model where they dump money in the top, and are able to extract more out the bottom. In all cases it didn't last more than a couple years at most.
For any small business considering hiring an ad agency that claims to have mastered this model ask yourself this: Why aren't they doing it themselves?
I do a lot of Facebook and most of my clients target CAC earnback in 7-9 months. I once was able to do arbitrage for my product and made a non-infinitely scalable positive ROI within one week with targeted Facebook ads but it was 2012. Not nowadays, the CPGs finally jumped in and the CPMs/CPCs are too competitive.
I think now the move is to go after territories that the major brands/agencies are afraid of. Tik Tok? I don't know, but I would trust doing DIY and examining the real numbers, I used to sell agency services, I'd say anything to win business. And 30% of the time we actually overdelivered on my absurd promises.
Small companies probably see the biggest bang for the buck on SEO and maybe blog posts. I feel like most small companies have niche offerings, since most things that can be big, are big by now. When people/business have issues with their sewage system, they go searching online for someone to help. So the difference in ranking between Walters Waste Water Warriors and Steve's Sewage System Saviors comes down who has the nicest, most informative site.
When I last worked in a consumer tech startup, our best ROI by far was paying tech youtubers or doing targeted ads to tech categories on youtube. Being in a startup I was vaguely aware of marketing efforts but I was never involved directly, so I can't say why or how that was discovered. All I know is my coworkers dealing with marketing said the roi and conversion rates specifically for youtube far outstripped traditional targeted tech ads through google or facebook. Of course this may have been because we weren't great at advertising, so your mileage may vary.
I suspect it was probably because of trust. An openly gifted reviewer gives more trustworthiness than just video ads or underhanded shills.
Given that you were probably smaller and not commodity priced to move I guess that sort of "honesty" helped move it better if people could closer to try before they buy.
I'm deeply suspicious that an entire profession has changed the way they behave in just 5 years. Moreover, I think you're ignoring a few things.
|
One is that companies can only spend money on marketing they have cash to invest. In a sudden downturn, cash may become scarce. Another other is that in extreme circumstances, panic is a common reaction. Even if the marketing director is perfectly rational, getting an ad budget may not happen if the CEO and the CFO are waking up in the middle of the night trying hard to figure how to keep the company from bankruptcy.
And a third, and possibly the biggest, is that it may be perfectly rational to pull ads now. There are whole classes of things that people can't really consume right now. All travel. All entertainment. For many, anything they have to leave the house to get, including big-ticket items like cars. And consumer psychology in a recession drives people to minimize discretionary purchasing, stick with familiar brands, and be more skeptical of anything new or unfamiliar.
It's also important to realize that advertising is an arms race. A lot of advertising spending is only required because other people are also spending. E.g., everybody in the world already knows what Coca Cola is; they advertise not to inform, but to maintain dominance. So it's perfectly plausible that a lot of places will cut spending and see nothing change because their competitors cut spending too.
The other thing you have to remember is ego. I see plenty of ads for WeWork office clones still running! Is it total neglect? I think it's hubris and Marketing Director's fighting to keep their ad spend, because if their entire budget is cut, guess who gets cut next?
Just left a data engineering job in adtech. Speaking from personal experience, marketers make terrible ROI analysts because they always want to spend the same or more. Agencies are especially bad.
Most incremental lift studies I've seen are valid, looking at CTR and A/B testing as you mentioned. But as soon as you start looking at revenue and brand lift, all studies seem to come to the same conclusion - the higher the spend, the better.
Don't get me started on multi-channel attribution... garbage.
Marketing is still very much spray and pray with diminishing returns. And in most cases, marketers will claim otherwise.
I got tired of drinking the koolaid. Modern marketing is just as fraudulent as ever. Agencies are just getting better at lying.
Agencies are thieves. At WPP Google wouldn't play ball with kickbacks, but you can't be a credible digital agency and not do search. So Google got their taste, just no GDN. Facebook wouldn't do kickbacks, so they were heavily discouraged.
Any DSP that could advance client goals but wasn't part of the inner circle would be overruled by senior management. Total gaslighting.
Everything goes to AppNexus or AppNexus partners. Because the parent company owns 20% or so of AppNexus. I got downvoted yesterday for this but I'll say it again, the mafia is in the tech, the tech is in the mafia. I have so many stories.
edit: The funniest bit was when I would take thoughtful, younger clients out to dinner who understood the business model. They would laugh and say "don't tell me too much, just hit the artificial KPIs and make me look good for my Christmas bonus.".
Speaking from personal experience, marketers make terrible ROI analysts because they always want to spend the same or more.
Yeah, principal-agent problem. At a previous job there was a case where we sent partial refunds to advertisers because we noticed we hadn't delivered as many impressions as they had paid for. Some of them were upset that by that, and would have preferred that we had just kept the full amount.
>What you're describing may have been the case, in a number of companies, five years ago, but it's just not meaningful now-days. Companies spend marketing dollars because marketing dollars work.
Companies do tons of things that don't make financial sense. And metrics can paint 100 pictures, including the picture the advertiser wants to paint. Not to mention it's in the best interest of the marketing department of a company and the advertising liaison within the company to continue spending money in advertising whether it works or not, if they want to have a job...
You're both right, _and_ wrong, because people, firms, and their respective behaviors vary a ton. Some marketing teams are great at measuring the effect of their ad spend. Some are not. It depends on the specific circumstances and the specific humans involved.
I'd be interested in an argument over whether more marketers are competent versus not, but any argument that presupposes uniformity among marketers is just silly.
I've worked in the field and have been closely involved in online advertising spending and you're way overstating this. Confusion, misdirection, dodgy attribution, questionable ROI calculations and outright fraud are rampant.
Furthermore I'd say most people with deep knowledge of the field (assuming they don't have a vested interest in arguing otherwise) would agree in a heartbeat.
Furthermore I'd say most people with deep knowledge of the field (assuming they don't have a vested interest in arguing otherwise) would agree in a heartbeat.
I don't know if this rhetorical device has a name, but basically saying "and other smart people would agree with me unless they had a sneaky reason not to" doesn't seem like supportive evidence of your point.. (... and I suspect most people would agree in a heartbeat.. sorry! ;-))
Marketing dollars work, and bidding for ad placement is competitive. The field is made of clever people that optimize it. However I think it's dangerous to believe that advertising is "numerical" or metrics can fully predict ROI of advertising - the same way subprime mortgages weren't numerical, and the same way a population mortality graph cannot predict coronavirus.
A sudden drop in revenue may adjust the economy of bidding and show a potential bubble. The tiny budgets and one click campaigns (with platforms inventing unique metrics & targeting woo) in social media marketing compared to traditional marketing (huge barrier to entry) feel like penny stocks vs traditional stocks. You can easily mislead individuals and tiny businesses.
I've fought to bring all those things to any marketing effort I've been involved with and have all but literally laughed out of the room. From my vantage point at least, the average marketing effort is "We spent all our budget, good job us."
ah but how do you really know if any of those numbers are causal? marketers need to account for multi-touch attribution, if a customer sees an ad on one platform and converts on another, or if they do it ten minutes outside of your conversion window.
there certainly are a lot of numbers in adtech and marketing. whether or not those actually relate to causal impact, across the industry, is not definitively answered. very few players (on the supply and demand side) have the scale and ability to properly do incrementality testing.
personally, i think people don't want to know the real results. if the numbers say it's not broken, why risk your job?
google and facebook are not "most players" -- they own walled gardens for measurement, it's much harder to do measurement on open web auction placements.
the google page you linked also says, "you won't be able to continue using this model if your data drops below 10,000 clicks on Google Search or below 400 conversions for the conversion action within 30 days." depends on what your market/CTA is, but a lot of SMB likely don't hit these thresholds. similarly with FB, my understanding is that you have to go through an account rep to set it up and that the minimum spend is fairly high to qualify.
i think that these measurements are possible to do with a reasonable level of certainty, but they require a particular set of circumstances to pull off.
Those are real concerns that I have no insight on if they are accounted for. I know that they can be accounted for and have been. TV and newspaper ads have been doing it for years where for obvious reasons they can't track where or when you saw the ad. McDonald's runs ads on those platforms anyway.
yeah you can try, for print/OTT the most common way to estimate lift is with DMA segmentation, compare lift in the DMA's which saw the ad versus the DMA's which didn't. but how do you pick the right DMA's to pair, etc. it's hard and i think most people don't do it right; again, it requires very large n to do correctly, which requires very large budgets.
mcdonalds is actually one of the savvier players here, they know that it's hard to quantify brand advertising if you only go off in-store purchases, so they spent very heavily to promote their app and develop a consumer-brand relationship there. now, if you see a mcdonalds ad on youtube, they can (try to) match your youtube ad impression to a purchase done through the app / credit card / other PII.
> You might not like marketing, but it's silly for the conclusion to be "thus, the entire field is made of people who don't know what they are doing".
It's not silly. Personally, I believe just that. Just because marketers use "conversion rates, A/B tests, attribution, and a lot of other techniques", doesn't mean they actually "calculate the ROI of advertising". They may be - and I believe are - mostly bullshitting themselves. Attribution is a hard problem. Doing an A/B test correctly is a specialized skill that requires some understanding of statistics most people in the industry don't have. And even the tools don't help, as evidenced by the famous case of Optimizely designing their product in a way that made people do A/B tests wrong - and in the exactly wrong way that made them feel Optimizely is helping them[0].
You can't seriously claim that all these small companies are hiring top STEM grads to trace ad spend and verify the work of top STEM grads working in small ad agencies.
You're both right. Marketing dollars work, but there's also a lot of wasted spend.
Many big campaigns are just brand awareness, or are poorly implemented with lots of supply-chain and measurement overhead competing for the cheapest impressions. Video content is increasing and leading to an oversupply of inventory. There's competition from connected TV and streaming services. Privacy regulations are affecting targeting and measurement.
A lot of budgets are also just paused in response to consumer demand, with uncertainty around the economy and shifting strategies to conserve cash and move to more efficient ads.
I have seen a lot of shoddy statistics used by ad people to justify the necessity of ad people. I would trust a random engineer over the ad team for this task purely as a matter of incentives.
So what is your experience in the business? I do not have any personal experience but I know plenty of people who work or have worked in adtech (e.g. ad words optimization consultants, ad agencies, affiliates who buy adwords) and from their stories I would say you overestimate the ad industry.
I also know of several (3 that I can think of on top of my head) companies who went bankrupt from buying inefficient ads.
What you're describing may have been the case, in a number of companies, five years ago, but it's just not meaningful now-days. Companies spend marketing dollars because marketing dollars work.
Any other interpretation is ideology-driven wishful thinking. You might not like marketing, but it's silly for the conclusion to be "thus, the entire field is made of people who don't know what they are doing".