Day 18-19

Today

• • Debrief on hypothesis testing chapter from ThinkStats

• The analytical approach to hypothesis testing and the connection back to simulation

• Project work time

For Next Time

• • Work on project

The materials for today are located in an ipython notebook under DataScience16/inclass/day18.

Some resources to help answer people's questions:

1. Why p = .05?

Other questions:

• 1. Why chi-squared versus sum of absolute deviations?

  2. When it says that statistical significance doesn't always mean that an effect is important or significant, is it saying that correlation does not imply causation?

1. How do you justify that the test statistic that you've chosen is a good one? Presumably, there isn't necessarily an immediately obvious test statistic for every problem. 

   1. Statistical Significance is pretty interesting, but I wonder how scientists who observe phenomenon actually make compelling arguments/theorem based upon that as a metric. For example, we could claim that the relationship between ice cream sales and crime is statistically signicant, implying the relationship is "more than just chance." Now, imagine a scientists who observes that the chemicl content of bubbles in hydrothermal vents is strongly correlated with the time of day that the sample was taken. We know that ice cream and crime is a silly relationship, but even experts in their field may not know that checmicl properties and time of day may not be related. 

   2. I suppose my question is, what else is there that can give us true certainty about phenomenon? Is there anything?

   3. What is the difference in Pearson's and Spearman's correlation? When do we know when to use which?

2. Would like to have in-class activity relating null hypothesis testing to our projects, maybe multiple ways of utilizing it in machine learning when looking at results and how to improve using this?

3. I know that the probability values that determine statistical significance are based off of areas under a normal bell curve, but I don't know the specifics of it. Can we go over that in class using a visual representation?