2015
O'Reilly Media / Infinite Skills
Lisa Qian
1:17
English

Deciding whether or not to launch a new product or feature is a resource management bet for any Internet business. Conducting rigorous online A/B tests flattens the risk. Drawing on her experience at Airbnb, data scientist Lisa Qian offers a practical ten-step guide to designing and executing statistically sound A/B tests.
- Discover best practices for defining test goals and hypotheses
- Learn to identify controls, treatments, key metrics, and data collection needs
- Understand the role of appropriate logging in data collection
- Determine how to frame your tests (size of difference detection, visitor sample size, etc.)
- Master the importance of testing for systematic biases
- Run power tests to determine how much data to collect
- Learn how experimenting on logged out users can introduce bias
- Understand when cannibalization is an issue and how to deal with it
- Review accepted A/B testing tools (Google Analytics, Vanity, Unbounce, among others)
Lisa Qian focuses on search and discovery at Airbnb. She has a PhD in Applied Physics from Stanford University.


01. A/B Testing, A Data Science Perspective
0101 Overview of the Course
0102 Why Should You Run A/B Tests?
0103 The 10 Steps and An Overview of Case Studies
0104 Case Study 1: Red vs. Green Button
0105 Case Study 2: Testing a New Landing Page
0106 Case Study 3: Price Recommendations on an Online Marketpl
0107 Summary: Setting Up an A/B Test
0108 Some of Your Options
0109 Scaling A/B Testing and Developing a Culture of Experimentation

Download File Size:330.43 MB

---