Estimating regression coefficients

The total length of the videos in this section is approximately 75 minutes. Feel free to do this in multiple sittings. You will also spend time answering short questions while completing this section.

You can also view all the videos in this section at the YouTube playlist linked here.

Question 1: If the estimated slope is 2, the sample mean of X is 1, and the sample mean of Y is 5, what is the estimated intercept of the regression line?

Measuring the relationship between two variables

Question 2: If you calculate that sample correlation between income (in dollars) and age (in years) is 0.5 (making this up), and then you decide to report income in terms of thousands of dollars and recalculate, what will the sample correlation be equal to?

Using correlation to estimate slope

Question 3: What is the estimated slope of the line, as described at the end of the video?

Justifying slope estimate as standardized correlation

Question 4: If you estimated the slope of the linear regression line connecting mean income (in dollars) and age (in years) is 1000 (making this up), and then you decide to report income in terms of thousands of dollars and recalculate, what will the new estimated slope be equal to?

There are several more questions about the relationship between slope and correlation toward the end of this module, but for now I want to move on.

In this module, I am giving you several ways to understand why we calculate the estimated slope the way we do. Above, you learned about the first way to understand: slope is a standardized version of correlation. Below, you will learn about several more ways to understand how we justify the slope estimator. The videos below were recorded previously, and I had presented correlation as a justification last instead of first. So, if I am counting the different ways to understand, and I seem to have forgotten that we already talked about correlation, that is why!

Residuals

Question 5: Which is bigger, on average, the residual of a data point given a separate means model v. the residual of a data point given an equal means model?

Justifying slope estimate by minimizing sums of squared residuals

Question 6: If the slope and intercept are selected to minimize the sum of squared residuals, which points can have a large influence on the choice of line? Describe the most influential points in a few words.

Justifying slope estimate by noticing that it is unbiased 

Question 7: Consider the following simulation. We choose the following model for the mean: Mean(Y|X) = 2+3*X. We set the sample size to n=3, with X values 1, 2, and 3. Then, repeatedly, we simulate the values of Y for these three data points by calculating Mean(Y|X) and adding (standard normal, independent) noise. For each simulated set of Y's, we estimate the slope of the regression of Y on X. We do this 1000 times, generating 1000 estimated slopes. What do we expect to see as the mean of these 1000 estimated slopes?

Justifying slope estimate by noting that it is an MLE

Question 8: What is "maximum likelihood estimation"? Try to describe it in just a few words.

My favorite way to justify the slope estimate - a weighted average of slopes

Question 9: Why would we multiply and divide by (Xi-Xbar)?

• If we rearrange the terms, this expression will now relate to the equation for the slope of a line

• If we rearrange the terms, this expression will now relate to the equation for the intercept of a line

More on the slope-to-center interpretation

Question 10: What would happen if we calculated the slope between each point and the mean point and then took the unweighted average of these slopes?

• The new estimate would be more influenced by points with extreme values of X

• The new estimate would be the same

• The new estimate would be less influenced by points with extreme values of X

Now we will return to the earlier interpretation and ponder some questions about slope and correlation, to make sure you leave this lecture solid on this concept.

Question 11: X is height, measured in inches, and Y is weight, measured in pounds. Given a data set, we estimate correlation=.6 and slope=5. How do you interpret the slope?

• For each 1 inch increase in height, weight increases by 5 pounds.

• For each 1 pound increase in weight, height increases by 5 feet.

Question 12: Continuing the previous problem. Suppose that we decide to record the weights in ounces instead of pounds. There are 16 ounces in a pound. Now what is the correlation? Now what is the slope?

Question 13: Again continuing the same problem. Now we decide to record weights in terms of standard deviations from the mean weight. The standard deviation of the weight, in pounds, is 33.5. Now what is the correlation? Now what is the slope?

Question 14: Last question about the same problem. Continuing to record weight in terms of standard deviations, we decide to also record height in terms of standard deviations. The standard deviation of heights, in inches, is 4. Now what is the correlation? Now what is the slope?

Question 15 (Review from previous module as a reminder): Which are the assumptions underlying regression/ANOVA? Check all that apply.

• The equation for the model for the mean is correct

• The values of Y are normally distributed around the predicted mean of Y for each value of X

• The variance of the residuals (the distances between the Y values and the predicted mean of Y) is the same for each value of X.

• The points in our data set are sampled independently of one another (including a particular data point does not tell us anything about whether another data point will be included).

That's all, folks!