That's wrong. There was a study [1] on this exactly that found that most people (in the US I think) who search products start with Amazon and not Google. In one of the recent Amazon earning reports, they reported $10B advertisement revenue, which is another indication that direct product search on Amazon is huge.
Actually Google is getting shittier and crappier every day. Btw. I usually look for things on Amazon, they search engine has the same issue as Google's. Only spam (they call it ad) in the first N positions. I would seriously pay for a service which has no PII data collection (still can measure the effectiveness of your engine), no ads and relevant results.
Is that still true? For most products Amazon search seems much improved to me, it used to be quite poor. Plus on mobile people may be biased to search via the app.
Most people don't even know what an URL is. They can't tell the difference between an app and a website. They think google is the internet. Hell, even the ones that do know better type the site URL into google to go to the site they want.
So yes, they have a huge potential.
They always have sucked doing things IRL, so it's not a certain win. But owning the road to the competition shop is powerful.
Not to mention they can force the hand of small shops: use our API or go down in ranking.