B2C giants like Microsoft, Facebook, and Google are the face of A/B testing. They run experiments with millions of users and enjoy overwhelming statistical power. Yet surprisingly, one-third of Statsig’s customers are B2B companies. This includes well-known names like Notion and Atlassian, as well as countless smaller B2B companies using experimentation to drive impact. B2B experimentation largely follows the same principles as B2C, but it faces unique statistical challenges—small sample sizes, skewed distributions, heterogeneous effects, complex metric selection, and ID mapping. This talk will explore what we’ve learned from our B2B customers, and how advanced techniques like stratified sampling, winsorization, and unit ID resolution can help. These challenges aren’t limited to B2B; they also appear in B2C testing at the edges. That means the solutions we discuss will be broadly useful to anyone working in online experimentation. Whether you’re running tests at scale or working with smaller datasets, this talk will help you make better, more reliable decisions with A/B testing.
Technical level: Technical practitioner
Session Length: 40 minutes
Supported by
