The previous sections in this chapter have focused on linear regression as a tool for summarizing trends in data and making predictions. These numerical summaries are analogous to the methods discussed in Chapter 2 for displaying and summarizing data. Regression is also used to make inferences about a population. The same ideas covered in Chapters 6-8 about using data from a sample to draw inferences about population parameters can also apply to regression. Previously, the goal was to draw inference about a population parameter such as µ or p.  In regression, the population parameter of interest is typically the slope parameter β. Inference about the intercept term is rare, but can be done where the vertical intercept is meaningful.

• r is an unbiased point estimate of the population correlation, ρ
• a is an unbiased estimate for the Y-intercept, β₀
• b is an unbiased estimate for slope, β₁
• ŷ is an unbiased estimate for mean response, µ_y

So a set of ordered pairs (x, y) used when fitting a least squares regression line are assumed to have been sampled from a population in which the relationship between the explanatory and response variables with the following equation:

Y = β₀ + β₁X + ε

Where ε, the error, is assumed to have a normal distribution with mean 0 and standard deviation σ
(ε ∼ N(0, σ)).

Inference for regression Assumptions:

Like any inferential technique, we need to meet an underlying set of assumptions in order to appropriately use them.

1. Like always, the data are collected from a well-designed, random sample or randomized experiment.
2. The relationship is Linear
3. The standard deviation of y is the same for all values of x.
4. The response y varies Normally around its mean
5. The residual errors are independent (no pattern).

You can check most of these using a residual plot.

Regression Standard Errors

Statistical software is typically used to obtain t-statistics and p-values for inference with regression, since using the formulas for calculating standard error can be cumbersome. However, the formulas are displayed below

The standard error of the residuals is:

Notice n-2 degrees of freedom!

Using that to calculate the standard error of the sampling distribution of the slope, b:

We can use this to do perform inference on the slope.

Inference on the slope

We now know the sampling distribution of the slope, b, is:

Hypothesis tests and confidence intervals for regression parameters have the same basic form as tests and intervals about population means.

Hypothesis tests on the slope

We often times want to check for the significance of the slope of a regression equation.  In other words is the slope actually different from 0?  In this case your Null Hypothesis would be:

H₀ : β₁ = 0

We can use the test statistic: