Testing Hypotheses¶

Data scientists are often faced with yes-no questions about the world. You have seen some examples of such questions in this course:

• Is chocolate good for you?
• Did water from the Broad Street pump cause cholera?
• Have the demographics in California changed over the past decade?

Whether we answer questions like these depends on the data we have. Census data about California can settle questions about demographics with hardly any uncertainty about the answer. We know that Broad Street pump water was contaminated by waste from cholera victims, so we can make a pretty good guess about whether it caused cholera.

Whether chocolate or any other treatment is good for you will almost certainly have to be decided by medical experts, but an initial step consists of using data science to analyze data from studies and randomized experiments.

In this chapter, we will try to answer such yes-no questions, basing our conclusions on samples and empirical distributions. We will start with an example from a study conducted in 2010 by the Americal Civil Liberties Union (ACLU) of Northern California.