Online experimentation has quickly emerged as a powerful tool in product development. It enables tech and other industries to run large-scale experiments and analyze results in near-real-time. This helps to prevent disasters and offer deep insight into the causal impact of product changes on user behavior.
In this session, we’ll discuss, demonstrate, and share code for the methodologies you can expect to see utilized in experimentation at companies like Facebook, AirBnB, OpenAI, and Google – and to a quickly growing degree at even the smallest startups. While the math dates back decades or even a century, the application of techniques like regression-adjustment, stratification, and heterogenous effect detection to enterprise-scale, real-world environments remains an evolving field.
Attendees will be given an overview of current statistical techniques, followed by hands-on demonstrations and code examples teaching them to learn variance reduction and other methodologies used to increase the power and reliability of large-scale randomized controlled trials.
Technical level: Technical practitioner
Session Length: 40 minutes
Supported by
