A/B testing: Setting up for success & choosing the right Statistical Framework by Benjamin Pettit from Cleo
We are all familiar with the benefit that A/B testing can have on businesses’ success, whether that’s improving customer experience or increasing revenue on websites and apps. In this session we’ll talk through tips we’ve learned from running a multitude of A/B tests at Cleo, from design through to implementation and analysis. The concepts of ‘holdouts’ in testing will be introduced, with a focus on how they can help teams track long term impacts to KPIs. Lastly we’ll look at Bayesian vs Frequentist frameworks, the benefits and pitfalls of each, and how we’re exploiting Bayesian statistics to get the most out of our A/B tests at Cleo.
Supported by
