"One key: continued experimenting with AI, even if an initial project doesn’t yield a big payoff. The authors say the most successful companies learn from early uses of AI and adapt their business practices based on the results. Among those that did this most effectively, 73 percent say they see returns on their investments. Companies where employees work closely with AI algorithms—learning from them but also helping to improve them—also fared better, the report found.
“The people that are really getting value are stepping back and letting the machine tell them what they can do differently,” says Sam Ransbotham, a professor at Boston College who coauthored the report. He says there is no simple formula for seeing a return on investment, but adds that “the gist is not blindly applying” AI to a business’s processes."
As Robin Hanson puts it, 'automation as colonization wave'. Just like electricity or computers or telephones or remote working: they always underperform initially because of the stickiness of organizations and bureaucracies.
Per Conway's law, no organization wants to reorganize itself to use a new technology like software, they want to make the new technology an imitation of itself, old wine in 5% more efficient new skins. It takes either intense near-death-experiences (see: remote working/corona) to force a shift, or starting up new businesses or units to be born-digital.
Those who undergo the birth trauma, like Google, are sitting on a river of AI gold and putting AI into everything; those who fail will whine on surveys about how AI is a scam and overhyped and an AI winter will hit Real Soon Now Just You Wait (it will pop any second now, just like how the media has regularly reported since 2009 about how the Big Tech bubble would pop)...
>those who fail will whine on surveys about how AI is a scam and overhyped and an AI winter will hit Real Soon Now Just You Wait
isn't this very similar to the logic you used a paragraph above though when you spun it as "all great things have birth problems, just wait?". The born-again AI company rhetorically sounds more like conversation to a Christian cult than a business strategy.
This issue of huge promises of the digital revolution followed by very meagre productivity gains actually has played out not just in 'AI' but a lot of sectors over the last three decades at this point.
Even for self-proclaimed AI companies like Google, how much of Google's financial bottom line is this new-agey wave of AI, and how much of it is pagerank, a ton of backend engineering and selling ads?
Googles search results and ad targeting are now entirely powered by machine learning. So basically all of their revenue is now from ML. PageRank is ancient stuff.
>all of their revenue is now from ML. PageRank is ancient stuff.
Seems like they would be making the same money either way, so the ML advantage might not be the major factor.
Conway's Law really does seem to show here.
If you replace occurrences of AI in the article with Natural Intelligence that is what so many companies really need to implement more of beforehand.
That way any decision-making that is delegated to AI afterward will not have the same limitations that the organization already had.
You usually don't want a business model with an anti-growth pattern baked into an even more opaque and unchangeable feature.
Problem is, deep application of the NI is going to get you most of the way you want to go business-wise, after that the AI might be even more difficult to justify.
Industrial-wise, with unique & complex equipment & data, where an operator gains skill through familiarity with both, a machine can be trained to gain some of that skill.
But there will always need to be someone better trained than the machine in order to get the most out of the machine.
The best investment is often going to be in a system to leverage their ability rather than try to operate without them at all.
I think that Google would be a memory without them aggressively developing and leveraging new tech (AI) for search and other ajoining services (maps, translate, adwords).
"Googles search results and ad targeting are now entirely powered by machine learning. So basically all of their revenue is now from ML. PageRank is ancient stuff."
I wonder if that is the reason I feel Google search has gotten so much more dull.
because the born-again stuff made me think of Evangelicals, but I guess if you want to go deep into this I think there's actually a lot of unironical parallels. The singularity is kind of like the rapture for nerds, AI people get weirdly dualistic about uploading their minds to clouds, and the overlap between SV capitalism and Protestantism goes back to Weber. I think it was Charles Stross who also pointed to similarities bwetween Russian Orthodox Cosmism and Western sci-fi. Secular western people in particular get weirdly religious about AI, like some sort of psychological replacement
For that matter, it seems that the analogy is to Evangelicals, who aren't generally considered to be a cult. They're too numerous, and while they do have many charismatic leaders, a cult usually has one particular top leader.
They do, however, practice the zeal of the converted, which is I think what they really were aiming it. And that by itself would qualify as "cultlike" by some definitions.
It’s because the ML solutions to optimization problems that are relevant to most businesses are garbage. Look at the attempts to solve the traveling salesman problem with just 100 nodes (a problem pretty much solved for the operations research community) with RL [1].
In other applications like forecasting, there is so much hype, and when you put the (non-interpretable) solutions to the test you end up disappointed (versus a pure stats approach).
There are of course fields where ML is pretty much the only viable solution, but trying to blame corporate bureaucracy to explain its failures in other fields does not help.
At this point another AI winter seems impossible. AI has such a strong foothold in functionality like voice recognition or image manipulation that are core to big tech company products, it wouldn't make sense for them to give up on the whole paradigm like what happened in the 80's.
While I don't think we'll have another AI winter for the reasons you point out, I do expect a large contraction in the AI market where a few of the successful companies in AI dominate and you won't see people throwing AI at problems that don't need it anymore.
In the 90s every washing machine had "fuzzy logic", it was the new hyped thing. Of course there are legitimate applications of fuzzy logic, but you don't have to apply it everywhere. It quickly died down once people noticed this fact.
Right now we're in the phase of the hype cycle where clueless managers ask their engineers to apply AI to anything, because they read about the AI revolution everyday and fear being left behind. But some problems have good non-AI solutions where AI won't reap much benefits, while the data collection, the experts, the time to develop, and the necessary restructuring to become an AI company costs a lot.
> Those who undergo the birth trauma, like Google, are sitting on a river of AI gold and putting AI into everything; those who fail will whine on surveys about how AI is a scam and overhyped and an AI winter will hit Real Soon Now Just You Wait
I wouldn't call Google as sitting on a river of AI gold.
It is using AI to make more gold from the same mine, instead of the obvious gold, it scratches and extract gold from the dirt and rocks. But it's still doing it from that same mine.
And that's exactly what AI is, garbage in, garbage out. I used to be a machine learning engineer in a company that is about as old as the dinosaur. They wanted to apply that gold digging AI machine google had on a landfill, which didn't quite pan out the way they liked it.
Not a machine-learning engineer per se, but I work with some machine-learning technologies at Google. Perhaps that's when you know that AI has truly become embedded in the fabric of a company: when it's become just another tool that ordinary engineers are expected to know and use to solve problems.
Anyway, I think AI is a sustaining innovation, not a disruptive innovation. It makes existing businesses work better, but it doesn't create new markets where none previously existed like the Internet did. Google makes a ton of money off AI; the core of the ads system is a massive machine-learning model that optimizes ad clicks like no human could. But that only works because they already have the traffic and the data, which they got by positioning themselves as the center of the Internet when it was young.
I do agree that companies need to adjust their processes to take full advantage of AI rather than expecting it to be a magic bullet, but I don't know if I really think "birth trauma" is the right metaphor. More like adolescence; it's a painful identity shift, and some people never successfully make the leap. Those who can't don't die, though, they just become emotionally stunted adults that never reach their full potential.
Search was pretty actively against AI usage at the time I left it in 2014. Much of this was because of Amit Singhal, though: he had a strong belief that search should be debuggable, and if there was a query that didn't work you should be able to look into the scores and understand why. There were AI prototypes that actually gave better results than the existing ranking algorithms, but weren't placed into production because of concerns on maintainability. I have no idea if this changed after Amit left.
I work on Assistant now, since recently rejoining Google, and it uses AI for the usual suspects: speech and NLP.
Google uses a ton of AI because they have a ton of data that’s easily categorized and has clear success criteria. Without those things, it’s doubtful that AI would’ve done Jack squat for google.
I have a whole smattering of thoughts based on the article and your comment...
From the article: "The authors say the most successful companies learn from early uses of AI and adapt their business practices based on the results."
This sounds exactly like the advice that ERP companies used to recommend to organizations. Don't try to customize the ERP, adapt your business to the out-of-the-box best practices. Those who don't often face huge costs and pain trying to adapt the tool to the business, rather than the other way around.
Interestingly, this might say more about organizational culture that encourages adaptation than anything else. It has been said (I think Steven Hawking may have been the one) that "adaptability is intelligence" [1].
From your comment: "Per Conway's law, no organization wants to reorganize itself to use a new technology like software, they want to make the new technology an imitation of itself, old wine in 5% more efficient new skins. It takes either intense near-death-experiences (see: remote working/corona) to force a shift, or starting up new businesses or units to be born-digital."
Conway's Law is about how the design of an organization's information systems often mirror their communication patterns. I have seen this myself to be quite realistic, anecdotally... although.. isn't this really kind of sad in a way? First of all, I see this as an excuse to write off groups of people as "legacy" and "outdated" - for example, digital transformation seems to be as much about a generational shift as it is a technological one. Or perhaps it is just showing the way - change culture/communication - value adaptability - to change systems.
Laws were meant to be broken, so why the belief that because of Conway's law, it always has to be this way, or that Conway's law is a constraint that cannot be broken? Certainly, we won't change things if we aren't willing to adapt.
Second, regarding the recognition of it taking near-death experiences to force change - doesn't that seem true for the broader human population, not just organizations? For that matter, why do we seem to be so stubborn and unable to flip these odds in our favor? That part is frustrating.
Next, to starting new business or units to be "born-digital" - of course, I understand why people think this way, why it probably works, and why it is a go-to strategy. But there is some part of me that this is part of disposable culture, it is partly exactly what is wrong with how we approach things - instead of incremental improvement, it is burn things down or start from scratch. Because we can't adapt the designs we implement, we have to start all over.. and it seems like a wasteful exercise. Where is our logical "exnovation" (the opposite of innovation)?
Lastly, about the gap between those who become "AI-enabled" companies and actually achieve success, and those who do not. In full recognition of the reality that those who do not get with the times, often get left behind, when it comes to technology - this gap worries me more. I'm thinking here of intellectual property, and that what is the best AI model will most likely always be locked and controlled by the profit motive - and thinking, what then?
I suppose this is why some thinkers are so worried about an AI-enabled future (ex. Elon Musk or Stephen Hawking) - it's not the positive potential that scares them, it's what happens to humans [2].
A lot of this is because of a lack of data literacy throughout the org and leadership.
Instead AI is thought of magic, something you hire a data scientist and they just do. Few in product, leadership, and other disciplines can think critically about different approaches to provide the right kind of data driven (and data skeptical) leadership.
I think it’s similar to how software was treated 20 years ago. You hired for it and effectively delegated it. You could supposedly manage software devs without technical knowledge. Now code literacy is heavily sought after in many disciplines. We need to get to the same place with data.
I worked for a data analytics consultancy for about 6 months, filled with data scientist types - the statistics and decision trees type.
One of the bigger projects was looking at computer vision to get a robot to detect when a pipe was busted underwater.
I asked the CTO about it and they said they were going to do it by teaching a mini off the shelf robot to locate a beer bottle in the office and transfer learn from there. I mentioned that would be highly unlikely for several reasons and was quickly ushered away from their pet project.
Suffice to say they eventually had to negotiate the final project scope down just a bit to offline binary image classification (broken vs. not broken).
So even a company filled with so called data scientists can fall into the trap of believing the snake oil.
Addendum: not necessarily believing their own snake oil I suppose, totally possible that they were just naive about the capabilities after reading too many wired/ars articles.
Less CTO AI Strongman, more underhanded/nuanced pressure in the form of this is a big pitch we could win and we're a small start up about to be acquired by a big conglomerate and it could affect how much we make from that deal from the MD and founders etc.
Edit: I only joined the company because I thought they were above all the snake oil, turned out their oil was just a different color.
This is my big concern with autonomous driving. Namely, we have confused perpetually being on the cusp of solving a big problem (self-driving) because we’ve solved an important subset of that problem (perception)
This feels pretty spot on for me. I’m actually very curious about the transition in software from very top down styles of leadership to more autonomous work. In my experience, middle managers will fight tooth and nail to ensure their own job security. I wonder how data scientists could pull off a similar feat.
AI has enormous potential that isn't being tapped by the average end-user company. And IMO the biggest reason for that is because these problems aren't AI problems, they're product management, UI/UX, development problems. If you have high quality user interaction, you can harness real quality data. Better data usually beats better algorithms. Not to mention all the time and effort saved in feature engineering. Instead of being a synchronized effort, it often feels like companies just take their useless slop and hand it over to their data scientists expecting them to churn out black magic.
And even if you don't get useful data, at the very least you can soften out a lot of the issues with integrating AI in a product. A few of the AI products I've seen give way too little control to the users and instead rely on opaque algorithms. Which is a very frustrating experience especially because those products will often charge a premium for the AI and try to create products where they wall you into their ecosystem and so it's very difficult to maybe export to a different solution that might give the user more control.
Fully Agree with this insight. Superior UI/UX will make a crappy model look like the magical future we imagine. When you flip on a snapchat face filter nobody thinks about the quality or accuracy of the object detection, facial landmark localization and the active shape model that's doing the lifting. I guess this is what they call the "AI effect."
The vast majority of what companies brand as “AI” is nothing more than basic BI reporting with some fancy marketing. The disappointment comes more from the realization that the “AI” that was purchased or deployed is just this.
Like other popular terms (“Big Data”, “Blockchain” ...) companies fear being left behind so they scrape up whatever they were already doing and get the marketing team to just say “now with AI” and carry on doing what they always did/sold.
I worked as consultant for a company, they basically had a database filled with data but they had no way of visualizing it. I quickly threw in an open source MySQL data vizualizer and pitched it as a quick solution until they figure out their larger needs. They were sold by another team with adding ML and AI to their solution and I was tasked with helping with the implementation.
After three months of discussions explanations and meetings and all the jazz, I asked them for a summary of the proposed solutions on their side and the costs as we were already making no progress and the cost estimates they were throwing around were massive. At the end, the cost estimate was huge, and the solution was basically "we're gonna try doing X and at the end it might work, or it might not work because what we do is magic".
So when talking to the CEO of the company whether we should embark on this expensive journey and decided against it. But because they had no other solution, they went with the vizualization tool I added and suddenly they realized it fits all their needs. Since half a dozen years, they've been using just that tool and extracting insane value from it.
I guess I lost my train of thought there, but to conclude, I believe until execs and managers learn to use the tools already available at their disposal, and generate awesome reports with almost 0 SQL knowledge - that can really cover pretty much every traditional business scenario I've encountered so far -, AI and ML are very much outside of their grasp and is more useful for technology businesses.
What a great story! And congratulations on getting a great tool into use.
Way back, I was getting to know about stock market analysis. I concluded that the recommended (technical) analysis was, by any individual recommender, just outside their area of mathematical competence. Someone with very little math would be impressed with (say) moving averages, but someone who understood some statistics would denigrate moving averages, but be impressed by Bollinger bands, etc.
Since no one understands neural networks :-) everyone was impressed by them.
Seems like a similar thing with AI/ML -- "Hey, this is beyond my level of understanding, it must be magic! Buy buy buy!"
This is a great insight. My company recently started working with agricultural analysts, and your example about moving averages is spot on. Initially, they also wanted the “magic” of NNs, and fortunately, after several prototypes, they understood that what they need is much simpler. As a result, after a couple of years, we’re starting to actually apply those NNs productively, having solved the simpler problems.
Basically by convincing them that to make "the real AI" work, they would need a lot of high-quality data that they wouldn't be able to produce. After several iterations with less data-hungry statistical methods they finally realized that they need a real product real soon (they were some kind of innovation department within a large corporation) and realized that the thing they need most is just a centralized GIS with basic computational capabilities.
Having successfully built that, those capabilities could be applied at scale and then we started experiments with more advanced analyses, this time more successful since both we had much more data and the customer became familiar with the data-intensive development.
Sounds about right. AI is for when you have already tried everything else and you still can't solve a well understood problem. Not "when you have a bunch of data and want to extract value".
This resonates with me. I was tasked in my organization to apply ML to most of the processes and sub teams..just because. People hand me over spreadsheets with 10 or 100 records expecting black magic. Many companies need to create a stream line process before they embark into the ML path. And once they decide to take that trip from what I have seen is that hiring ML engineers and DS, may not make sense at the very beginning. What I believe is that path from Analytics, then API/black box products from Cloud vendors selling "MLaaS" where they use tons of data to offer solutions in specific areas, such as finance, manufacturing, etc. From there pivot and continue to invest more in ML if it makes sense.
Check out https://github.com/metabase/metabase . I'm not clear on their open source vs paid model, but I'm pretty sure you can host it yourself and their core is open source.
Just like any truly transformative tech, AI takes long-term planning, tons of competence and picking the parts that make most sense for your current business goals. So obviously AI fails for most organizations because they lack all of these things and the fact that most never operate in scale or tackle problems difficult enough to need AI.
Maybe one solution is finding the AI equivalent of Microsoft Office, ie a group of cheap tools that are so powerful, flexible and integrated that they can be used by individual employees and teams across all industries and business needs.
I agree that AI is very difficult and requires hard work by competent operators to be successful but I think there’s a deeper problem - most business data is irrelevant noise. It won’t matter how talented your DS team is if there is no signal in your data relevant to business operations for the ML model to identify.
Completely agree. Most business data is noise and most of the signals are already discovered as simple rules and heuristics. On the other hand, if you have a strong signal in your data, even a simple algorithm like linear/logistic regression will be able to help. What I’ll call “signal hunting” is probably the best use of DS resources and also the hardest thing to do.
I’ve done my share of experiments with ML/AI and where I’ve seen the most interesting value has been NLP applications (such as categorizing customer comments or assigning categories to products based in description) and finding “factors that influence behavior x” which then can be turned into either a model or a few simple rules.
I'm convinced this is one of the main sources of inefficiency in modern management, whether using AI or not. Managers are incentivized to be action prone so as to demonstrate their value continuously, and they are also expected to put in processes. Often these processes are on way too tight of a cadence and what ends up happening is that managers spend their entire time evaluating and chasing noise.
Azure ML Studio is a good tool for your non-data-science developer to play around with ML tools. I picked it up at my company and it was fairly easy to make models and predictions with.
Of course, once you actually start to get good at it you want to switch back to using code, but it's a good way to start.
How do we know that AI is truly transformative tech? The null hypothesis is that most businesses could achieve equivalent ROI by using older, simpler data analysis techniques like linear regression.
Because most large tech companies are driven by it right now and you likely interact with a system influenced by ML every time you use your phone, email, maps, translation, social media etc etc. The catch it's only transformative if you're dealing with large enough problems. You're right that more tiny companies should probably stick to classic analysis techniques.
I would argue that linear regression is under the umbrella of ML. And probably the most important ML technique for existing businesses. A lot of the "low hanging fruit" for existing business cases is not necessarily discovering how to use regression analysis on existing data (which is probably already happening at a lot of places) but how to operationalize and continually update linear regression models so you can make them a mission-critical part of the infrastructure.
Linear regression has been around since before computers. It's a real stretch to put that under the umbrella of AI. Nor is AI required to continually update linear regression models; a "for" loop will suffice.
That loop has to run over high quality, relevant data. The regression model needs to be retrained if the production data drifts from the original training dataset. The output of the regression model needs to be wired into a downstream application or decision support tool. That downstream application needs to know how to deal with error bars. A data scientist / statistician needs to re-architect the whole thing if the data or business requirement changes substantially.
As usual, the hardest problems are outside of the code.
I agree with you here Data and measurement is the single most important part of the process.
From my experience working in an industrial plant which has been involved in several machine learning trials a lot of the time there are attempts made to use complex modeling techniques to make up for a lack of measurements.
Something I question is whether the outcome would have been better if the money which was invested into hiring AI consultants was spent on better plant instrumentation.
Industrial Instruments are not cheap something like a PGNAA analyser (https://en.wikipedia.org/wiki/Prompt_gamma_neutron_activatio...) is an expensive capital purchase and I suspect some people have unrealistic expectations that AI and machine learning can replace things like this.
I think there is some middle ground where AI complements better sensors (maybe instrument companies should be pushing this). I've yet to see any of the data experts push back and say something like "actually you need to measure this better first before we can model it."
Most ML algorithms boil down to linear regression with some loss function. Even deep learning networks are essentially just linear regressions stacked on top of each other, with lower layers being trained on features predicted by higher layers.
I think if neural networks or SVMs are AI, then linear regression is as well. Neural Turing machines and other recent developments I think are closer to the layperson's idea of "AI," though.
>Most ML algorithms boil down to linear regression
I think (particularly with DL) it would probably be more accurate to claim it boils down to nonlinear logistic regression rather than linear regression. To your point, both are relatively old techniques
Nonlinear regression is still linear regression with transformed features. Kernel regression, for example, just uses features generated from the data using a supplied kernel function/covariance function. DL just allows the features to be trained.
You could replace "AI" there with any other newly developed tech paradigms, like "Big Data" several years ago. Your null hypothesis is probably correct for most companies, but there will always be some companies for whom the new tech is beneficial. Most companies do not need big data, and are fine running a single postgres instance for the entirety of their lifetime, just like most companies probably don't need AI.
This is true, but I think the real benefit of the “Big Data” paradigm (scare quotes included) was the spreading the idea that you should actually measure things, and then make a decision. You’d think this was obvious, but apparently it wasn’t. (Think back to the book Lean Startup, whose premise is, “A/B test hypotheses.”) Similarly, the “AI Revolution”, could be boiled down to, “run a regression.”
First - there is much more data now than 5 or 10 years ago; it is generated by every process and is easy to store.
Second - there is a greater art and capability to aggregate and manipulate data. It's simply faster, but also there is a lot of supporting technology in the form of workflows and tooling.
Third - there are more algorithms now; these are often derived from AI research (DNN, RNN, Bayesian things..)
The first two definitely mean that linear regression can generate much more value than 10 years ago.
The third one is a product of the frustration with linear regression and many other "traditional" algorithms. In many domains (speech, images, text processing) the community smashed its head on the wall for 30 years before the computational resources and algorithmic tricks that came out in 2010->now came on stream. You just can't do much with TFIDF or similar with text - I tried very very hard; on the other hand using a transformer is like bloody magic.
Doesn't the first part of your comment contradict the second? If it does require a lot of long-term planning and competence, maybe the last thing you want to do is to distribute it to indivudal employees.
I suspect that it's hierarchical in nature, in the end. That is, you have the aspects of "using AI" that are big, complex, enterprise-wide and do require lots of long-term planning and a strategic focus. But there are also the "low hanging fruit" where you can utilize the "MS Office of AI" to achieve smaller, more tactical goals.
I feel like I use a ton of tools everyday that would have been considered "AI" 10 years ago, like content-aware fill in Photoshop, translation and correcting software, etc.
I've used a quote from Deming before, and I'll use it again.
Deming, from Out of the Crisis (1986):
People with master's degrees in statistical theory accept
jobs in industry and government to work with computers. It is
a vicious cycle. Statisticians do not know what statistical
work is, and are satisfied to work with computers. People
that hire statisticians likewise have no knowledge about
statistical work, and somehow suppose that computers are the
answer. Statisticians and management thus misguide each other
and keep the vicious cycle rolling. (p. 133)
The last time I used this in the context of data scientists. Now it's AI. I'm seeing this at my employer now as well. They think AI will save them, and want to use it for so many things. But the problem is that most people don't really know what to do with it, and most work won't really benefit from it (it could, but not the way they're going about it which is mostly throwing buzzwords at the wall). It was the same problem with statisticians and with data scientists.
Statisticians could help us do our work better, but not the way they did it. Data scientists could help us do our work better, but not the way they did it. AI will have the same problem for most businesses who choose to follow trends and fads rather than evaluate the actual value and utility of the technology and subject matter. And will be of greatest benefit to those who hire experienced people. This was the problem with both statisticians and data scientists for many companies. They'd hire to fill a slot, not for expertise.
In many ways it's mirrored in companies trying to transition to DevOps as well. Filling a slot. Rather than comprehending whether the approach is valid for them, what the approach actually entails, and properly evaluating who should fill the relevant positions.
Probably even less if you exclude benefits from AI that could be achieved with more pedestrian solutions but now that AI is hot they are suddenly not as flashy on powerpoints. Even then I think a lot of those projects have their benefits severely exaggarated because people who propose them then feel pressed to show benefits or embarass themselves. Even if you exclude those projects, I suspect a lot of projects are simply to "get ball in the game", to build a capability even before there is real need to be able to react to competition. Now, the value of that capability doesn't seem so much, this probably leads to many cut projects.
Still, probably beats blockchain. There was a time when I heard a blockchain project being awarded almost every other month at the companies I worked for and I still wait for ANY single one of them to do anything useful.
I don't get many things that they need to smash AI into. but I do get it more than the blockchain hype. oh my god if mother and it's dog had a blockchain startup. what where the benefits I have no fucking idea. I like thinking of blockchain as the free energy equivalent in the software world.
Unsurprising. It's a hype bubble; it seems like only VCs and executives haven't realized that yet.
It has however become a very good metric for spotting low quality technical leadership. If an executive or similar is talking about "AI" or "Machine Learning" without the ability to identify specific use cases that they're hoping to implement, then that is a huge red flag. AI, as much as it actually exists, is a tool to be applied to an end, not magic pixie dust to be sprinkled onto your product to make it more profitable.
I agree with this at face value, but I want to push back a bit so that I can sharpen my own argument. How can I argue that "AI" is a hype bubble to a layman while simultaneously acknowledging that we've seen the development of self-driving cars, music identification, Siri and Google Assistant, search by image, etc.?
Obviously this isn't all "AI" in the historical CS sense of the word. It's a mix of progress in things like signal processing, computer vision, NLP, neural nets and transformers, and the raw computer engineering that has made all of that practical on modern hardware.
Are you and I sticks-in-the-mud for not just going along with calling this "AI"? Is there actual worthwhile nuance by calling things their proper names rather than just kowtowing to the word that normal people have latched onto?
1) It’s entirely probable that self driving cars are over hyped, and that it’ll be decades before we hit level IV or V self driving cars. Lane keeping is nice, but it’s hardly revolutionary.
2) Audio and image identification is indeed an area where great strides have been made, and if those are relevant to your business, then you’re in great luck, and you should absolutely leverage that. But this goes back toward my “it should be aimed at a specific use case” statement; image identification is much much more narrow than how AI is being hyped.
3) Personally I’m less than impressed with the progress that voice assistants have made; they present great, but they have utterly failed to make a dent in my day to day living. It seems like end users regularly oscillate between “this is amazing”, “this is stupid”, and “this is incredibly creepy”. In my opinion this area saw a great big leap forward about 5-6 years ago and progress has basically stopped.
If you read back about older AI bubbles, what you’ll find is that there are bits and pieces left over from those bubbles that are unquestionably improvements on what came before. But critically these improvements fell well short of the grandiose promises made by AI boosters at the time. It’s entirely probable that this AI boom will follow a similar pattern; a massive cycle of hype and collapse that leaves behind useful techniques and tools that nonetheless fall well short of what was promised during the cycle.
As far as what is and is not considered to be “AI”; that definition has changed readily over the years. Ground breaking techniques are regularly called “AI” until they stop being called that and get new names. We’re already seeing this process work now with ML.
I think there are the AI leaders, and then there is everyone else. What is the difference?
The leaders are mostly big tech, who have driven the step-change advances you describe. This, IMHO, was due to their pre-existing mastery of data. They already had a ton of well-organized data, because they were engineering cultures, and data was the lifeblood of their business (ads, search, shopping). Once ML/AI came into the picture, it was full steam ahead.
Most others are (blind) followers, and cannot tease apart the engineering bit from the (data) science hype. They get fixated on the latter (data science and ML/AI) and forget the engineering, or "scale" part.
The first question should not be about AI/ML, but on the other hand, do you have solid (data) engineering where your data is easily accessible to any data scientist? By now it should be apparent that "data is the new oil" and will be useful even if you don't plan to do deep learning.
If you don't have solid (data) engineering and "data at scale" for anyone, anywhere, then your ML/AI efforts are doomed.
It seems to me the hype bubble is centered around things that require GAI to solve. Progress in more general methods for weak AIs has led to confusion about what is possible with current methods.
In the traditional CS sense, the things you mention could all be considered weak AI.
I predict as we continue to transition to stronger AIs with more advanced capabilities we will continue to see a shift in what people think of as “AI”.
Sorry but it's just wrong to call it a hype bubble because there are real and tangible uses of ML in production today on massive scaled systems that give extreme outsized results. All of GAAFM rely on ML at scale at this point for a large subset of their products.
It's just that these results are also EXTREMELY unevenly distributed - and good luck breaking into anything that can use applied ML and not get crushed by GAAFM.
However I agree with your point here:
>If an executive or similar is talking about "AI" or "Machine Learning" without the ability to identify specific use cases that they're hoping to implement, then that is a huge red flag.
So it's not as simple as "AI is Hype" - it's not hype - it's just that most organizations will struggle to actually implement it because all the data/talent/compute etc... sits in GAAFM.
It’s a hype bubble due to the delta between what’s being promised, a revolution of basically every aspect of business, and what’s delivered, massive improvements in very specific domains where a large data set is available and easily classified.
It’s unquestionable that there are certainly areas that ML is delivering in spades, but it’s nowhere near as ubiquitous as the hype implies.
Even for the behemoth companies that are able to harness AI, it seems like the domains are a) heavily constrained b) fault tolerant. For example, voice assistants - they have very limited capabilities and consumers will accept pretty poor performance. Look at the errors in Google's attempts to automatically answer questions in searches.
Do you have any examples of domains where a FAANG has operationalized AI/ML outside of consumer products?
Any insight into the actual methodology? I couldn't find specifics, but I would be curious what their baseline condition is.
I wonder if the baseline case is "no control optimization" or if it was based on current control best-practices. For example, one article claims it produces cooler water temperature than normal based on outside conditions. This is a best practice in good energy management through wet-bulb outdoor air temperature reset strategies without using ML. If their 40% savings was above and beyond these best practices, that's a pretty big accomplishment. If it's based on the static temperature setpoint scenario (i.e. non best practice), it's less so.
Edit: after skimming [1], it seems like their baseline condition was the naive/non-best practice approach. I'm not discounting the potential for ML, but I think a more accurate comparison should use traditional "best practice" control strategies, not a naive baseline condition. In some cases, it seems like the ML approach identified would be less advantageous than current non-ML best-practices (e.g., increasing cooling tower water by a static 3deg rather than tracking with a wet-bulb temperature offset)
"In fact, the model’s first recommendation for achieving maximum energy conservation was to shut down the entire facility, which, strictly speaking, wasn’t inaccurate but wasn’t particularly helpful either."
> if you showed a google home to someone in 1980s they would be absolutely floored.
I am not so sure about that. If you came from a time machine and said "This is an AI from the year 2020", they would try and converse with it and quickly realize it's unable to converse. People from the 80's would probably assume by the year 2020 they'd have sentient robots and be disappointed when all it can do is turn on the lights when asked a specific way.
Well FAANG all produce consumer products, so that wipes out a bazillion legitimate applications, but you've still got that Facebook and Google sell ads, which uses AI for targeting. Data centre cooling was already mentioned, but did you know lithography now uses ML? There's even work on using ML for place and route.
The VC business model seems to be mostly be "finding the greater fool during IPO", they probably know very well that most of those things are buzzwords.
This is sort of like all new tech and domain application. The application tree first grows in breadth, then depth. I'm guessing real use cases will take 5-20 years to settle.
It’s like all new tech, if you fall into the survivorship bias trap and only look at the ones that have survived.
Personally I think this will be another “AI winter”, where we will gain some improvements in narrow domains but fall far short of what was promised, leading to disillusionment and a reduction in research budgets.
Like all hype bubbles except blockchain, AI will be big in 10 years, it’s just hype because it’s too early. Web 2.0, e-commerce, mobile, etc all grew into their hype.
Funny thing I: I added some AI to my CS curriculum in Uni. They said back then: "20 years ago, they made a lot of AI predictions, but they obviously did not happen, but 20 years from _now_ : oh boy". That was 20 years ago, and well, history seems to repeat itself.
We'll see. Maybe with Moore's law alikes pushing the processing capacities and a separate 10x improvement in the mechanics (ie 'how' AI learns).
Machine translation, search, recommendation, face/object detection, image processing are just a subset of very real technologies broadly deployed today that benefit from machine learning (a subset of AI).
AI does suffer from the problem where the term seems to be defined as “what we can't do now”. That said, I think those are also good examples of why it's important to be realistic – good translation has been significant, especially for travelers, but search, recommendations, face detection, etc. have been modest incremental improvements. That's great to have but a lot of people aren't content with that billing and create problems by overselling it as game-changing.
Exactly. - "In my experience and opinion, as soon as something is well defined it becomes Artificial Narrow Intelligence - Vision systems, Natural Language Processing, Machine Learning, Neural Networks - it loses the name "AI" and gets its own specific name."
Face detection is really valuable to me. I have cameras on the front of my house and I get an alert on my phone when someone unrecognized comes up to the house. It sometimes false triggers on me and my wife but it let us know 1) when a package is delivered 2) when our kids wander out to the front yard 3) hasn't happened yet, but if a sketchy person came onto our property.
Yes, and it's gotten better than the older CV approaches. My point is that it's nice but it didn't significantly transform your life the same way, say, a smartphone did.
It's funny how people attributed a ton of value to block chains and crypto currency, when most of crypto's value as an alternative to government fiat only came about because of its ability to mask illicit payments for drugs, weapons and contract killings. People went all in on Bitcoin in 2017, when the FBI had shut down Silk Road, the main Bitcoin mover for a long time, in 2014.
Two years ago all I heard in the health care industry was how awesome blockchain would be for our industry. Lots of presentations by big knockers in the company, big time investments and initiatives like our company was going to be the "tech leader" in the space, blah blah blah.
Two years on and you can't find any mention of it on any of the internal websites, all the PPT's and videos have been scrubbed from the NAS drives they were once promoting people to go watch and learn from. All the supposed "partnerships" have been dissolved, or never took off in the first place.
Two years later and blockchain is nothing but a memory at the company I work at. Just as fast as it arrived, it disappeared without so much as a trace of it ever existing in the first place.
VR isn't a hype bubble at all though. It's just that graphics tech has only recently gotten powerful enough to pull it off, but it's still expensive. It'll come down in price and get better in quality, while probably staying relatively small due to the setups needed. It's just like any high-end gaming setups. Most people are fine with simple gaming on their phone (Angry Birds or whatever is popular these days), some want consoles, some want powerful PCs, some are satisfied with Oculus Quest type VR devices, and some want quite powerful PCs for VR. VR is growing way too slowly and steadily to really characterize as "hype".
VR is already widely used in industrial training. Send a forklift driver through VR training and the safety lessons might stick a little bit better, so your insurance company will cut you a discount and it can pay for itself.
Indeed. That being said, I think it has even more potential in making more adaptable simulators of things that we couldn't simulate well or dynamically enough before.
Games are huge and, as a result, VR as a vehicle for delivering games will be huge. IMHO the world-changer is going to be AR, and the two will likely retain many complementary objectives in hardware/software/skillsets.
I feel the article conflates AI with the broader concepts of business intelligence and data science. If the “learning” is happening on the humans side, you’re essentially just collecting and analyzing data.
Or is the article literally suggesting the 10% of companies that profit off of AI are the 10% “learning” by retraining their models on newer data? To me that is akin to saying companies that maintain their websites tend to be more profitable, it’s not that much of a revelation.
Stuff like "mutual learning" is annoyingly general. I feel it's like the kind of business speak that's so high in the upper atmosphere as to be barely applicable on the ground.
I don't have the patience to suss out whether they're talking about a deep broad concept or just being too vague.
For instance:
> Organizational learning with AI is demanding. It requires humans and machines to not only work together but also learn from each other — over time, in the right way, and in the appropriate contexts. This cycle of mutual learning makes humans and machines smarter, more relevant, and more effective. Mutual learning between human and machine is essential to success with AI. But it’s difficult to achieve at scale.
I feel this is talking about a problem at my company -- I had a lot of stumbling blocks implementing AI predictions because the size of packages are stored as strings, weight information is inconsistent, etc. etc. so it's very hard to categorize things based on what's in the database.
Perhaps that's what they mean from "humans learning from machines and machines learning from humans" -- actually following decent data standards because you have to if you want to do anything with it -- but damn, just say that.
I feel this is a silly way of saying "maybe we should have actually listened to the programmers from the very beginning when they were talking about doing things the right way."
Honest question: is just collecting and analyzing data not considered a use case for AI? I mean sure some businesses can be essentially run by a recommender system or something. But it also seems valuable if you can do something like use NLP to get better quantification of customer feedback, which is really just collecting and analyzing data for a human to use later.
If it’s humans making observations in the data, changing (non AI parts) of their software stack, and reanalyzing the data it sounds like there’s no supervised or unsupervised machine learning going on, it could be, but the article is too hand wavy for me to be sure. That’s why I think a more accurate title is “companies realize most of the work in AI comes down to feature engineering and data prep, not algorithm design”
Absolutely. The current state of AI/ML practically requires a human to interpret the results. Even a recommender system is just collecting and analyzing data for humans to consume in some way. And that system needs to be maintained and retrained regularly to produce meaningful results.
wrong. I have trained plenty of unsupervised models where the inferred results are purely for human consumption. In fact I believe unsupervised models more often require a human to analyze the results.
Whether an algorithm is supervised or not certainly does affect whether you need to retrain it periodically. Also it does not at all affect whether the output is fed to end users or to other algorithms
Important context: this comes from BCG, a business strategy firm, not MIT. This paper speaks to executives, NOT people trying to build anything directly. Don't try and read from a tech perspective; it's way higher up in the clouds.
This helps answer the questions: is this worth investing in? What returns can I expect, and when? What types of operating model is needed to put this all together? How can this play alongside everything else I'm running/building/exploring?
I would be very happy if more executives were convinced by slick powerpoints that AI is bust.
Lord knows they don't listen to their technical staff, because they read in a glossy magazine that ML is the future, and only suckers don't have an "AI play".
More specifically, this comes from BCG Gamma, a division working specifically to create AI solutions for large corporations. It’s true, the article addresses executives, not engineers, but the content generally holds.
A lot of companies seem to have a poor understanding of what AI is. Lots treat it like an ancient soothsayer.
Heard too many stories of companies trying to use it to predict equipment failures or something along those lines. Did they have failures in their datasets? No. Obviously they did not succeed.
A lot of AI hype reminds me of the hype of data warehouses. The amazing predictions you were going to get. Turns out getting the data and/or using 'AI' is easier than asking the right questions and figuring out what to do when you get a particular result. Both of these tools are good at what they do. But the predictions of what they can do seem a bit off the mark. Even the failure modes seem to be mirroring a lot of what happened with data warehouses 'your data is not right' 'your data is in the wrong form' 'your filters were not right' 'you used the wrong topology' and so on.
Not saying anything bad here it is just an interesting observation of 'history repeating'.
>A lot of companies seem to have a poor understanding of what AI is.
Yes, this is true, but a lot of this poor understanding can be tracked down to the over-enthusiasm (and straight overselling speak, mind you) of most of the AI scientists.
The company I work for (mining) hired a top-notch AI guy to "change the fundaments of data management and processing in the company, using the latest AI technology". Guy left six months later... I still wonder why he lasted that long.
Not only that but good anomaly detection is easy to mess up you have to be careful with how you amplify the anomalies in the dataset so that you don't wind up predicting garbage anyway.
I am an AI skeptic hard-core, but I think I disagree with the premise. Just look a bit harder. AI is actually used in things like computer graphics (hello nvidia DLSS?) and hollywood special effects. Also its used in other entertainment. Does lectures on machine learning on youtube that collect massive amounts of money through patreon count? How about language translation? Aren't state of the art models used in language translation today? Does google use transformer today in google translate? I'm not sure but it seems like it could definitely work. Sure it seems most use cases are not profit-generating. Is that they key here? I'd say its more marketing related tech-feats. But does that mean there is no benefit?
The "key" is a total abuse of the term "AI". The dictionary definitions basically say "machines that appear intelligent", and "intelligence" is "ability to acquire knowledge [/skills through experience]". The bar is extremely low for what one can call AI. QuickSort qualifies. A* qualifies.
Getting AI in 2020 is like getting "a website" 1999. No idea what you really can do with it but it's cool to say you have it.
Though the true benefit is all of those companies that relabeled their trusty optimization algorithms to AI and got funded for having the correct buzzwords.
I don't agree with this notion. A website is (and was in 1999) infinitely more useful, immediately creating value for your company/brand. On the other hand, I don't even know what AI means these days.
If you consider many companies hiring their first applied statisticians due to the AI/DS hiring, then the gains are massive.
Yes, most companies probably do not know how to effectively leverage AI knowledge from 2012-2020 (deep learning era). But, classical ML/AI tools such linear regression, basic clustering, A* search, bayes nets etc. are older tools that many industries are now using for the first time.
It's always amusing to get updates from our AI/ML department (I'm tangentially involved with them enough to stay in the loop).
It's 3 people - smart, but fairly typical CS Masters level. They've been working on AI/Machine Learning models for 3 years, with a truly unreasonably large budget for hardware/software, and very little to show for it.
If the models they were developing actually worked, there would be no reason for my company to continue their primary business, because we would effectively predict the next 20 years of stock performance in a fairly large segment of the market.
At least the CFD people have gotten some good use out of the giant stack of GPUs they bought.
The problem with ML/AI is that it almost never provides people with a significant new capability. It can improve and augment, but rarely lets people do something that they couldn't do before. There is no "killer app"
It's very difficult to have a transformative impact by adding features or making a process 5% more efficient. It seems like we're very far away from AI enabling the average person to have new opportunities and experiences in the same way as past technological innovations have.
example: email enables me to communicate instantly and asynchronously with someone on the other side of the world. what does ml enable me to do?
I suppose I agree in the sense that it is, really, the killer app. Still I think OP's point still holds, machine translation is undeniably infinitely more practical with ML but it still is just an augmentation. Same going for voice recognition.
I think OP’s first paragraph sounds like a reasonable framework for viewing the utility of ML, though is a bit too black/white. Bit they lose me completely when they said that we’re “very far away from AI enabling the average person to have new opportunities and experiences in the same way as past technological innovations have”.
In any case, I agree with the thread’s larger point that the buzz and hype around the field is exhausting, and certainly not always warranted. Nothing makes my eyes roll more than reading a post from a data “thought-leader” on LinkedIn.
Many, many businesses would gladly take a 5% improvement in efficiency. That can translate to tens or hundreds of millions of dollars for very large companies.
The AI tools and hardware need to focus on being more plug and play and autoconfigurable as I’ve noticed my team spends way too much time on operational issues and it significantly slows down progress on machine learning and AI tasks.
This complaint happens with many new technologies.
In the 1980s companies poured lots of money into personal office computers with no measured increase in employee productivity. This was called thd productivity paradox.
Something about this new wave of AI tech makes me think about Bell Labs or Xerox Parc. It isn't always the companies researching that gain the significant benefits. I wonder if in 30 years we'll be watching VR documentaries on the entrepreneurs of the future waxing poetically about the tech they saw at Google/Facebook/Amazon and how it inspired them to create their trillion dollar tech giants.
And yet Climate Corporation completely changed agricultural crop insurance and basically took over Monsanto from the inside. When companies do make good use of AI it can completely change markets in ways that can be hard to fully understand.
I think time is probably running out for the current 2010s AI hype train. It happened before in the 80s/early nighties when few successful practical applications occurred and investment became disillusioned.
People are using deep learning as a political tool in most of the companies. It's a political gain if someone get a chance to work on deep learning in the company. And because of this only incompetent people work on AI in industry. I am only talking about practitioner here not the scientist.
Here's one simple suggestion - when creating documents, instead of using PDF or Word, why not use forms like plain text or similar to facilitate text mining. One of the biggest problems with knowledge management right now is dealing with machine-hostile formats like PDF.
This is far from an easy problem - I have been struggling with this over the past few years. tools like pdftotext only work with the most simply formatted documents. In contrast, the typical PDFs generated in many business organizations is almost impossible to extract text from, especially with insets, callouts and multiple columns. Amazon's Textract is one of the best PDF text extractors out there and even that struggles with many ordinary looking PDFs
If the data is fundamentally 1D then why is it arranged with insets and multiple columns ?
I think the answer is that it's arranged that way because that arrangement makes it easier for humans to interpret. Because humans prefer to see data in such 2D arrangements, file formats like PDF were developed so that 2D layouts could be defined and preserved across platforms.
Hence the fundamental problem here is that we want computers to be able to read data that is designed for human readability. That is not an easy problem and it is in fact a large part of what people mean when they talk about "AI".
I'd push that a step further and say the problem is about data that's been rendered into a different form for human readability. The plain serial text of a magazine article exists in principle and likely wasn't written with ideas like column splits and image-wrapping at the front of the author's mind. 2sk21 advocates shipping that serial form instead of something image-like. That's what's machine-hostile about PDF: it makes the least semantically significant parts easy for the machine to discern at the cost of obfuscating the really important ideas.
Though as a counterpoint, separating content from presentation is already a fairly common practice.
I think it depends a great deal on what type of data you're talking about. If it's a magazine article then yes it's essentially a stream of text and 2D formatting is probably secondary. However if it's tabular data then the 2D formatting is essential to its interpretation.
A lot of the text we encounter in the world does depend to a significant extent on 2D layout to facilitate its comprehension. Examples include many (perhaps most) websites, product labels, tax forms and receipts.
None of that is the fault of the PDF format which is intended to faithfully represent 2D layouts. As you mention, if the goal is accurate labeling of content other formats exist which are more suitable to that purpose (though I think the concept of separating content from presentation has always worked better in theory than in practice).
I'm not sure if this falls under "ML", "AI" or just "algorithms", but the useful "AI" being presented to our Company produces detailed geographic and structural reports on high res aerial photography captured by private planes.
e.g. highly detailed feature reports of properties and landscapes, applied to millions of acres of data.
Is the AI being a decision making agent? Maybe that would be more in the ML/algorithms area, but I’m not sure if there is a precise definition agreed upon here...
The most useful part of AI is the ETL and feature engineering. Those are also the parts that are now mostly commoditized. ML plays a minor role for most legacy companies. These companies should (and do) just white-label startup solutions as their own.
I feel like it sounds good not only to VCs, but also to consumers who assume AI will benefit them without truly understanding what it is and why they should use it.
What exactly is so great about AI anyways that people expect it to deliver major benefits? Every AI breakthrough in the news is the result of massive amounts of data and compute applied to very narrow problem, such as games or image labelling, and even then the AI models are very brittle and easy to break. What is the specific killer app for this kind of product? It's hard for me to think of anything.
Mortgage applications and insurance products are largely all computerized for some companies. Some home buying companies make offers on homes without any human intervention.
These pathetic statements from MIT articles have no value. It's just blog spam.
If you want to be pedantic, some companies have had benefit.
Scammy AI companies.
Other parts of the pyramid scheme like training organisations.
Chess websites
Massive data companies like Google and Baidu in only some areas of operations.
The rest is just data collection and people doing data science lying that they are doing AI, often with a ROI that's a loss.
“The people that are really getting value are stepping back and letting the machine tell them what they can do differently,” says Sam Ransbotham, a professor at Boston College who coauthored the report. He says there is no simple formula for seeing a return on investment, but adds that “the gist is not blindly applying” AI to a business’s processes."
As Robin Hanson puts it, 'automation as colonization wave'. Just like electricity or computers or telephones or remote working: they always underperform initially because of the stickiness of organizations and bureaucracies.
Per Conway's law, no organization wants to reorganize itself to use a new technology like software, they want to make the new technology an imitation of itself, old wine in 5% more efficient new skins. It takes either intense near-death-experiences (see: remote working/corona) to force a shift, or starting up new businesses or units to be born-digital.
Those who undergo the birth trauma, like Google, are sitting on a river of AI gold and putting AI into everything; those who fail will whine on surveys about how AI is a scam and overhyped and an AI winter will hit Real Soon Now Just You Wait (it will pop any second now, just like how the media has regularly reported since 2009 about how the Big Tech bubble would pop)...