12.5 The Regression Equation

We often want to use values of the independent variable to make predictions about the value of the dependent variable.  For example, we might want to use the amount a business spends on advertising each quarter to make a prediction about the revenue the business will generate that quarter.  When a linear relationship exists between an independent and dependent variable, we can build a linear model of that relationship, and then we can use that model to make predictions about the dependent variable.

Simple linear regression is a modeling technique in which the linear relationship between one independent variable [latex]x[/latex] and one dependent variable [latex]y[/latex] is approximated by a straight line, called the line-of-best-fit or least squares line.  It is important to note that the line-of-best-fit only models the linear relationship between the independent and dependent variables.

The equation for the regression line is:

[latex]\begin{eqnarray*} \hat{y} & = & b_0+b_1x \\ \\ \hat{y} & = & \mbox{predicted value of } y \\ x & = & \mbox{value of the independent variable} \\ b_0 & = &y\mbox{-interecept of the line} \\ b_1 & = & \mbox{slope of the line} \end{eqnarray*}[/latex]

The value of [latex]\hat{y}[/latex] is the estimated value of [latex]y[/latex].  It is the value of [latex]y[/latex] obtained using the regression line. It is not generally equal to the value of [latex]y[/latex] from the sample data.  The values for the slope [latex]b_1[/latex] and the [latex]y[/latex]-intercept [latex]b_0[/latex] in the line-of-best-fit are calculated using the sample data and the least squares method.  Although there are formulas to calculate the values of the slope and [latex]y[/latex]-intercept in the regression line, we will calculate the slope and [latex]y[/latex]-intercept using the built-in functions in Excel.

The slope of the linear regression equation:

• The slope of the line-of-best-fit [latex]b_1[/latex] and the correlation coefficient [latex]r[/latex] have the same sign.  That is, [latex]b_1[/latex] and [latex]r[/latex] are either both positive or both negative.
• The slope [latex]b_1[/latex] of the regression equation tells us how the dependent variable [latex]y[/latex] changes for a one unit increase in the independent variable [latex]x[/latex].
• When interpreting the slope, be specific to the context of the question, using the actual names of the variable and correct units.

The [latex]y[/latex]-intercept of the linear regression equation:

• The [latex]y[/latex]-intercept [latex]b_0[/latex] of the line-of-best-fit is the predicted value of the dependent variable [latex]y[/latex] when [latex]x=0[/latex].
• When interpreting the [latex]y[/latex]-intercept, be specific to the context of the question, using the actual names of the variable and correct units.

Making Predictions with the Linear Regression Equation

Given a specific value of the independent variable [latex]x[/latex], the linear regression equation may be used to predict/estimate the value of the dependent variable [latex]y[/latex].  To make predictions, the following condition must be met:

• There must be a linear relationship between the variables.  The stronger the linear relationship, the better the prediction will be.
• The linear regression equation is only valid to predict values of the dependent variable.  That is, we may only use the equation to solve for [latex]\hat{y}[/latex] for a given value of [latex]x[/latex], and not the other way around.
• The linear regression equation should only be used to make predictions for [latex]y[/latex] for values of [latex]x[/latex] within the domain of the [latex]x[/latex] values in the sample data used to construct the regression equation.  The regression equation does not provide reliable predictions for values of [latex]x[/latex] that fall outside the domain of  the [latex]x[/latex] values in the sample data.

Errors and The Least Squares Method

The difference between the actual value of the dependent variable [latex]y[/latex] (in the sample date) and the predicted value of the dependent variable [latex]\hat{y}[/latex] obtained from the linear regression equation is called the error or residual.

[latex]\begin{eqnarray*} \mbox{Error} & = & \mbox{Actual Value}-\mbox{Predicted Value} \\ & = & y-\hat{y}\end{eqnarray*}[/latex]

Graphically, the absolute value of the error is the vertical distance between the actual value of [latex]y[/latex] (the point on the scatter diagram) and the predicted value of [latex]\hat{y}[/latex] (the point on the linear regression line).  In other words, the absolute value of the error measures the vertical distance between the actual data point and the line.

The slope and [latex]y[/latex]-intercept for the linear regression equation are generated using the errors and the least squares method.  The idea behind finding the line-of-best-fit is based on the assumption that the data are scattered about a straight line.  For any line, the errors can be calculated, squared, and then these squared errors can be added up.  Of all of the possible lines, the line-of-best-fit is the one line that minimizes this sum of the squared errors.  Any other line will have a higher sum of the squared errors compared to the sum of the squared errors for the line-of-best-fit.

_{Watch this video: Slope and Intercept for Linear Regression in Excel by ExcelIsFun [18:29]}

Concept Review

A regression line, or a line-of best-fit, can be drawn on a scatter diagram and used to predict outcomes for the [latex]y[/latex] variable in a given data set or sample data.  Regression lines can be used to predict values within the given set of data, but should not be used to make predictions for values outside the set of data.

Attribution

“12.3 The Regression Equation“ and “12.5 Prediction“ in Introductory Statistics by OpenStax is licensed under a Creative Commons Attribution 4.0 International License.