I doubt there are companies exclusively rely on A/B tests to make decisions, but even if there were, someone has to think about hypothesis and design the right experiment to then confirm with data. You can't just have a computer running around and make experiments and confirm them for youº.
--
º: you kind of can, if you have enough fresh data, to search for and derive statistically significant results automatically.
Maybe it's just me, but I don't think the secret behind this kind of success is just properly designing experiments. Again, if this were true, the best founders would be data scientists. But they're not. They are visionaries. So I reckon it must be the vision.
Vision is obviously necessary - but a big problem folks run into is failing to recognize when something is not working. A/B testing is one way to keep yourself from falling into the trap of thinking 'I know better than my users what they should want'
--
º: you kind of can, if you have enough fresh data, to search for and derive statistically significant results automatically.