
课集5课

视频若干

案例若干
课程介绍
Google产品设计内训教程：A/B测试
A/B测试也被称之为分割测试，简单来说，就是为同一个目标制定两个方案（比如两个页面），让一部分用户使用 A 方案，另一部分用户使用 B 方案，记录下用户的使用情况，看哪个方案更符合设计。
本课程将介绍如何选择和表征指标来评估您的实验，帮助您合理的设计一个实验，并且用足够的统计力量，分析出正确的结论结果，以及如何确保实验的参与者得到充分的保护。
第1课：A/B测试概述
This lesson will cover what A/B testing is and what it can be used for. It will also cover an example A/B test from start to finish, including how to decide how long to run the experiment, how to construct a binomial confidence interval for the results, and how to decide whether the change is worth the launch cost.
第2课：实验条款概述
This lesson will cover how to make sure the participants of your experiments are adequately protected and what questions you should be asking regarding the ethicality of experiments. It will cover four main ethics principles to consider when designing experiments: the risk to the user, the potential benefits, what alternatives users have to participating in the experiment, and the sensitivity of the data being collected.
第3课：选择和检定指标
One of the most important and timeconsuming pieces of designing an A/B test is choosing and validating metrics to use in evaluating your experiment. This lesson will cover techniques for brainstorming metrics, what to do when you can't measure what you want directly, and characteristics you should consider when validating your metrics.
第4课：设计实验
This lesson will cover how to design an A/B test. This includes how to choose which users will be in your experiment and control group  different online definitions of a "user", and what effects different choices will have on your experiment. It will also cover when to limit your experiment to a subset of your entire user base, how to calculate how many events you will need in order to draw strong conclusions from your results, and how this translates into how long to run the experiment. Finally, the lesson will cover how various design decisions affect the size of your experiment, so you will know which decisions to revisit if you need results more quickly.
第5课：分析结果
This lesson will cover how to analyze the results of your experiments. Step one is always to run some sanity checks so that you can catch problems with your experiment setup. Then, you will learn how to check conclusions with multiple methods, including a hypothesis test on the effect size and a binomial sign test, if you get results that surprise you. You will also learn how measuring multiple metrics for the same experiment can make analysis difficult, and some techniques for handling multiple metrics. Finally, you will learn about several analysis "gotchas", and what to do if you see them, including how Simpson's Paradox can affect A/B tests, and why even statistically significant results might disappear when you launch.