Woah, that's a fun one to play around with. Pancake must be commonly used in idioms. Try deleting each word individually, it's pretty hard to get it down to pancake.
There is a different one based on wlc (So it's a wayland compositor, rather than just an X11 window manager). The one you linked doesn't seem to have been updated since 2015
A question for Patrick, or for anyone else with experience implementing A/B testing from the early stages of a service/product:
When and where do you start? If you start too early, you don't have enough traffic to be statistically significant.
In the article, Patrick said that he started with the trial signup form in his 3rd year.
Most of the advice on A/B testing that I've found is understandably aimed at people with existing business/products that can massively benefit from it. Does anyone have any more material about how to get started from the early stages?
This has been my mantra as well. Test big and you'll likely fail or win a lot faster.
I'd also suggest making sure you approach testing by starting with a hypothesis and then testing that hypothesis. By voicing or writing down your hypothesis, you will be more likely to avoid the "stupid test trap", a phrase I use to describe running tests like changing button colors -- what hypothesis is changing button colors trying to prove? That people like red more, "for reasons"?
As mentioned above, each A/B test you run has a serious opportunity cost. I'm fortunate enough to have thousands of people going through my sales funnel daily but, even then, a test will still take several weeks or months to validate and anything approaching a change of 5% or less will really eat away at your ability to test more meaningful things.
My experience is that most tests will come out under 5% differences. But most organizations will have some test that they could run with a greater than 15% win.
If your business scale does not allow you to run tests and detect the wins you hope for, you're better off following other people's best practices than you are trying to discover best practices from running your own tests.
Also note that most businesses have a "conversion funnel" where there are a number of steps from visitor to getting paid. If your business is big enough, you want to focus on getting paid. If your business is too small to get results that way, you should get started with just the first step in the funnel. That's what Patrick did with the trial signup form.
