{"id":427,"date":"2021-12-15T23:18:39","date_gmt":"2021-12-15T23:18:39","guid":{"rendered":"https:\/\/pressbooks.palomar.edu\/introtostats\/chapter\/chapter-13\/"},"modified":"2024-10-14T21:01:34","modified_gmt":"2024-10-14T21:01:34","slug":"chapter-13","status":"publish","type":"chapter","link":"https:\/\/pressbooks.palomar.edu\/introtostats\/chapter\/chapter-13\/","title":{"raw":"Chapter 13: Linear Regression","rendered":"Chapter 13: Linear Regression"},"content":{"raw":"
\n

Key Terms<\/h2>\n<\/header>\n
\n

curvilinear models<\/span><\/a><\/p>\n

error<\/span><\/a><\/p>\n

least squares error solution<\/span><\/a><\/p>\n

line of best fit<\/span><\/a><\/p>\n

mean squared error<\/span><\/a><\/p>\n

moderation models<\/span><\/a><\/p>\n

multiple regression<\/span><\/a><\/p>\n

residual<\/span><\/a><\/p>\n

standard error of the estimate<\/span><\/a><\/p>\n\n<\/div>\n<\/div>\n

In Chapter 11<\/span><\/a>, we learned about ANOVA, which involves a new way of looking at how our data are structured and the inferences we can draw from that. In Chapter 12<\/span><\/a>, we learned about correlations, which analyze two continuous variables at the same time to see if they systematically relate in a linear fashion. In this chapter, we will combine these two techniques in an analysis called simple linear regression, or regression for short. Regression uses the technique of variance partitioning from ANOVA to more formally assess the types of relationships looked at in correlations. Regression is the most general and most flexible analysis covered in this book, and we will only scratch the surface.<\/p>\n\n

Line of Best Fit<\/h3>\n

In correlations, we referred to a linear trend in the data. That is, we assumed that there was a straight line we could draw through the middle of our scatter plot that would represent the relationship between our two variables, X<\/span> and Y<\/span>. Regression involves solving for the equation of that line, which is called the line of best fit.<\/p>\n

The [pb_glossary id=\"715\"]<\/a>[\/pb_glossary]line of best fit<\/span> can be thought of as the central tendency of our scatter plot. The term best fit<\/span> means that the line is as close to all points (with each point representing both variables for a single person) in the scatter plot as possible, with a balance of scores above and below the line. This is the same idea as the mean, which has an equal weighting of scores above and below it and is the best singular descriptor of all our data points for a single variable.<\/p>\n

We have already seen many scatter plots in Chapter 2<\/span><\/a> and Chapter 12<\/span><\/a>, so we know by now that no scatter plot has points that form a perfectly straight line. Because of this, when we put a straight line through a scatter plot, it will not touch all of the points, and it may not even touch any! This will result in some distance between the line and each of the points it is supposed to represent, just like a mean has some distance between it and all of the individual scores in the dataset.<\/p>\n

The distances between the line of best fit and each individual data point go by two different names that mean the same thing: errors and residuals. The term [pb_glossary id=\"713\"]<\/a>[\/pb_glossary]error<\/span> in regression is closely aligned with the meaning of error in statistics (think standard error or sampling error); it does not mean that we did anything wrong, it simply means that there was some discrepancy or difference between what our analysis produced and the true value we are trying to get at. The term [pb_glossary id=\"719\"]<\/a>[\/pb_glossary]residual<\/span> is new to our study of statistics, and it takes on a very similar meaning in regression to what it means in everyday parlance: there is something left over. In regression, what is \u201cleft over\u201d\u2014that is, what makes up the residual\u2014is an imperfection in our ability to predict values of the Y<\/span> variable using our line. This definition brings us to one of the primary purposes of regression and the line of best fit: predicting\u00a0scores.<\/p>\n\n

Prediction<\/h3>\n

The goal of regression is the same as the goal of ANOVA: to take what we know about one variable (X<\/span>) and use it to explain our observed differences in another variable (Y<\/span>). In ANOVA, we talked about\u2014and tested for\u2014group mean differences, but in regression we do not have groups for our explanatory variable; we have a continuous variable, like in correlation. Because of this, our vocabulary will be a little bit different, but the process, logic, and end result are all the same.<\/p>\n

In regression, we most frequently talk about prediction, specifically predicting our outcome variable Y<\/span> from our explanatory variable X<\/span>, and we use the line of best fit to make our predictions. Let\u2019s take a look at the equation for the line, which is quite simple:<\/p>\n

Y\u0302<\/span> = a<\/span> + bX<\/span><\/p>\n

The terms in the equation are defined as:<\/p>\n

Y<\/span>\u0302<\/span>: the predicted value of Y<\/span> for an individual person<\/p>\n

a<\/span>: the intercept of the line<\/p>\n

b<\/span>: the slope of the line<\/p>\n

X<\/span>: the observed value of X<\/span> for an individual person<\/p>\n

What this shows us is that we will use our known value of X<\/span> for each person to predict the value of Y<\/span> for that person. The predicted value, Y\u0302<\/span> (\u201cY<\/span>-hat\u201d), is our best guess for what a person\u2019s score on the outcome is. Notice also that the form of the equation is very similar to very simple linear equations that you have likely encountered before and has only two parameter estimates: an intercept (where the line crosses the y<\/span>-axis) and a slope (how steep\u2014and the direction, positive or negative\u2014the line is). These are parameter estimates because, like everything else in statistics, we are interested in approximating the true value of the relationship in the population but can only ever estimate it using sample data. We will soon see that one of these parameters, the slope, is the focus of our hypothesis tests (the intercept is only there to make the math work out properly and is rarely interpretable). The formulas for these parameter estimates use very familiar values:<\/p>\n

\"\"<\/p>\n

<\/a>\"\"<\/p>\n

We have seen each of these before. \"\" and \"\" are the means of Y<\/span> and X<\/span>, respectively; \"\" is the covariance of X<\/span> and Y<\/span> we learned about with correlations; and \"\" is the variance of X<\/span>. The formula for the slope is very similar to the formula for a Pearson correlation coefficient; the only difference is that we are dividing by the variance of X<\/span> instead of the product of the standard deviations of X<\/span> and Y<\/span>. Because of this, our slope is scaled to the same scale as our X<\/span> variable and is no longer constrained to be between 0 and 1 in absolute value. This formula provides a clear definition of the slope of the line of best fit, and just like with correlation, this definitional formula can be simplified into a short computational formula for easier calculations. In this case, we are simply taking the sum of products and dividing by the sum of squares for X<\/span>.<\/p>\n

Notice that there is a third formula for the slope of the line that involves the correlation between X<\/span>\u00a0and Y<\/span>. This is because regression and correlation look for the same thing: a straight line through the middle of the data. The only difference between a regression coefficient in simple linear regression and a Pearson correlation coefficient is the scale. So, if you lack raw data but have summary information on the correlation and standard deviations for variables, you can still compute a slope, and therefore an intercept, for a line of best fit.<\/p>\n

It is important to point out that the Y<\/span> values in the equations for a<\/span> and b<\/span> are our observed Y<\/span>\u00a0values in the dataset, not<\/span> the predicted Y<\/span> values (Y<\/span>\u0302<\/span>) from our equation for the line of best fit. Thus, we will have three values for each person: the observed value of X<\/span> (X<\/span>), the observed value of Y<\/span> (Y<\/span>), and the predicted value of Y<\/span> (Y<\/span>\u0302<\/span>). You may be asking why we would try to predict Y<\/span> if we have an observed value of Y<\/span>, and that is a reasonable question. The answer has two explanations. First, we need to use known values of Y<\/span> to calculate the parameter estimates in our equation, and we use the difference between our observed values and predicted values (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) to see how accurate our equation is. And second, we often use regression to create a predictive model that we can then use to predict values of Y<\/span> for other people for whom we only have information on X<\/span>.<\/p>\n

Let\u2019s look at this from an applied example. Businesses often have more applicants for a job than they have openings available, so they want to know who among the applicants is most likely to be the best employee. There are many criteria that can be used, but one is a personality test for conscientiousness, with the belief being that more conscientious (more responsible) employees are better than less conscientious employees. A business might give their employees a personality inventory to assess conscientiousness and study existing performance data to look for a relationship. In this example, we have known values of the predictor (X<\/span>, conscientiousness) and outcome (Y<\/span>, job performance), so we can estimate an equation for a line of best fit and see how accurately conscientiousness predicts job performance, then use this equation to predict future job performance of applicants based only on their known values of conscientiousness from personality inventories given during the application process.<\/p>\n

The key in assessing whether a linear regression works well is the difference between our observed and known Y<\/span> values and our predicted Y<\/span>\u0302<\/span> values. As mentioned in passing above, we use subtraction to find the difference between them (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) in the same way we use subtraction for deviation scores and sums of squares. The value (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) is our residual, which, as defined above, is how close our line of best fit is to our actual values. We can visualize residuals to get a better sense of what they are by creating a scatter plot and overlaying a line of best fit on it, as shown in Figure 13.1<\/span><\/a>.<\/p>\n\n

\n
\n

<\/a>Figure 13.1.<\/span> Scatter plot with residuals. (\u201c<\/span>Scatter Plot with Residuals<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n\n<\/div>\n<\/div>\n

\n
\"\"<\/div>\n<\/div>\n

In Figure 13.1<\/span><\/a>, the triangular dots represent observations from each person on both X<\/span> and Y<\/span> and the dotted line is the line of best fit estimated by the equation Y<\/span>\u0302<\/span> = a<\/span> + bX<\/span>. For every person in the dataset, the line represents their predicted score. The brackets between the triangular dots and the predicted scores on the line of best fit are our residuals. (For ease of viewing, they are only drawn for four observations, but in reality there is one for every observation.) You can see that some residuals are positive and some are negative, and that some are very large and some are very small. This means that some predictions are very accurate and some are very inaccurate, and that some predictions overestimate values and some underestimate values. Across the entire dataset, the line of best fit is the one that minimizes the total (sum) value of all residuals. That is, although predictions at an individual level might be somewhat inaccurate, across our full sample and (theoretically) in future samples our total amount of error is as small as possible. We call this property of the line of best fit the [pb_glossary id=\"714\"]<\/a>[\/pb_glossary]least squares error solution<\/span>. This term means that the solution\u2014or equation\u2014of the line is the one that provides the smallest possible value of the squared errors (squared so that they can be summed, just like in standard deviation) relative to any other straight line we could draw through the data.<\/p>\n\n

Predicting Scores and Explaining Variance<\/h4>\n

We have now seen that the purpose of regression is twofold: we want to predict scores based on our line and, as stated earlier, explain variance in our observed Y<\/span> variable just like in ANOVA. These two purposes go hand in hand, and our ability to predict scores is literally our ability to explain variance. That is, if we cannot account for the variance in Y<\/span> based on X<\/span>, then we have no reason to use X<\/span> to predict future values of Y<\/span>.<\/p>\n

We know that the overall variance in Y<\/span> is a function of each score deviating from the mean of Y<\/span> (as in our calculation of variance and standard deviation). So, just like the brackets in Figure 13.1<\/span><\/a> representing residuals, given as (Y<\/span> \u2212 Y<\/span>\u0302<\/span>), we can visualize the overall variance as each score\u2019s distance from the overall mean of Y<\/span>, given as (\"\"), our normal deviation score. This is shown in Figure 13.2<\/span><\/a>.<\/p>\n\n

\n
\n

<\/a>Figure 13.2.<\/span> Scatter plot with residuals and deviation scores. (\u201c<\/span>Scatter Plot with Residuals and Deviation Scores<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n\n<\/div>\n<\/div>\n

\n
\"\"<\/div>\n<\/div>\n

In Figure 13.2<\/span><\/a>, the solid line is the mean of Y<\/span>, and the blue brackets are the deviation scores between our observed values of Y<\/span> and the mean of Y<\/span>. This represents the overall variance that we are trying to explain. Thus, the residuals and the deviation scores are the same type of idea: the distance between an observed score and a given line, either the line of best fit that gives predictions or the line representing the mean that serves as a baseline. The difference between these two values, which is the distance between the lines themselves, is our model\u2019s ability to predict scores above and beyond the baseline mean; that is, it is our model\u2019s ability to explain the variance we observe in Y<\/span> based on values of X<\/span>. If we have no ability to explain variance, then our line will be flat (the slope will be 0.00) and will be the same as the line representing the mean, and the distance between the lines will be 0.00 as well.<\/p>\n

We now have three pieces of information: the distance from the observed score to the mean, the distance from the observed score to the prediction line, and the distance from the prediction line to the mean. These are our three pieces of information needed to test our hypotheses about regression and to calculate effect sizes. They are our three sums of squares, just like in ANOVA. Our distance from the observed score to the mean is the sum of squares total, which we are trying to explain. Our distance from the observed score to the prediction line is our sum of squares error, or residual, which we are trying to minimize. Our distance from the prediction line to the mean is our sum of squares model, which is our observed effect and our ability to explain variance. Each of these will go into the ANOVA table to calculate our test statistic.<\/p>\n\n

ANOVA Table<\/h3>\n

Our ANOVA table in regression follows the exact same format as it did for ANOVA (hence the name). The top row (Model) is our observed effect, the middle row is our error, and the bottom row is our total. The columns take on the same interpretations as well: from left to right, we have our sums of squares, our degrees of freedom, our mean squares, and our F\u00a0<\/span>statistic.<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

\u2211<\/span>(Y<\/span>\u0302<\/span> \u2212 \"\"<\/span>)2<\/span><\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

\"\"<\/span><\/p>\n<\/td>\n

\n

\"\"<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

Error<\/p>\n<\/td>\n

\n

\u2211<\/span>(Y<\/span> \u2212 Y<\/span>\u0302<\/span>)2<\/span><\/p>\n<\/td>\n

\n

N <\/span>\u2212 2<\/p>\n<\/td>\n

\n

\"\"<\/span><\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

\n

\u2211<\/span>(\"\"<\/span>)2<\/span><\/p>\n<\/td>\n

\n

N <\/span>\u2212 1<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

As with ANOVA, getting the values for the SS<\/span> column is a straightforward but somewhat arduous process. First, you take the raw scores of X<\/span> and Y<\/span> and calculate the means, variances, and covariance using the sum of products table introduced in our chapter on correlations<\/span><\/a>. Next, you use the variance of X<\/span> and the covariance of X<\/span> and Y<\/span> to calculate the slope of the line, b<\/span>. (The formula for calculating <\/span>b<\/span><\/a> was provided earlier.) After that, you use the means and the slope to find the intercept, a<\/span>, which is given alongside b<\/span>. After that, you use the full prediction equation for the line of best fit to get predicted Y<\/span> scores (Y<\/span>\u0302<\/span>) for each person. Finally, you use the observed Y<\/span> scores, predicted Y<\/span> scores, and mean of Y<\/span> (\"\") to find the appropriate deviation scores for each person for each sum of squares source in the table and sum them to get the sum of squares model, sum of squares error, and sum of squares total. As with ANOVA, you won\u2019t be required to compute the SS<\/span> values by hand, but you will need to know what they represent and how they fit together.<\/p>\n

The other columns in the ANOVA table are all familiar. The degrees of freedom column still has N <\/span>\u2212 1 for our total, but now we have N <\/span>\u2212 2 for our error degrees of freedom and 1 for our model degrees of freedom; this is because simple linear regression only has one predictor, so our degrees of freedom for the model is always 1 and does not change. The total degrees of freedom must still be the sum of the other two, so our degrees of freedom error will always be N <\/span>\u2212 2 for simple linear regression. The mean square columns are still the SS<\/span> column divided by the d<\/span>f<\/span> column, and the test statistic F <\/span>is still the ratio of the mean squares. Based on this, it is now explicitly clear that not only do regression and ANOVA have the same goal but they are, in fact, the same analysis entirely. The only difference is the type of data we feed into the predictor side of the equations: continuous for regression and categorical for ANOVA.<\/p>\n\n

Hypothesis Testing in Regression<\/h3>\n

Regression, like all other analyses, will test a null hypothesis in our data. In regression, we are interested in predicting Y<\/span> scores and explaining variance using a line, the slope of which is what allows us to get closer to our observed scores than the mean of Y<\/span> can. Thus, our hypotheses concern the slope of the line, which is estimated in the prediction equation by b<\/span> (the slope of a population, as opposed to b<\/span>, which is the slope of a sample). Specifically, we want to test that the slope is not zero:<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

A non-zero slope indicates that we can explain values in Y<\/span> based on X<\/span> and therefore predict future values of Y<\/span> based on X<\/span>. Our alternative hypotheses are analogous to those in correlation: positive relationships have values above zero, negative relationships have values below zero, and two-tailed tests are possible. Just like ANOVA, we will test the significance of this relationship using the F\u00a0<\/span>statistic calculated in our ANOVA table compared to a critical value from the F\u00a0<\/span>distribution table. Let\u2019s take a look at an example and regression in action.<\/p>\n

Example <\/span> Happiness and Well-being<\/p>\n

Researchers are interested in explaining differences in how happy people are based on how healthy people are. They gather data on each of these variables from 18 people and fit a linear regression model to explain the variance. We will follow the four-step hypothesis-testing procedure to see if there is a relationship between these variables that is statistically significant.<\/p>\n\n

Step 1:<\/span> State the Hypotheses<\/h5>\n

The null hypothesis in regression states that there is no relationship between our variables. The alternative states that there is a relationship, but because our research description did not explicitly state a direction of the relationship, we will use a non-directional hypothesis.<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n\n

Step 2:<\/span> Find the Critical Value<\/h5>\n

Because regression and ANOVA are the same analysis, our critical value for regression will come from the same place: the F\u00a0<\/span>distribution table, which uses two types of degrees of freedom. We saw in the ANOVA table<\/span><\/a> that the degrees of freedom for our numerator\u2014the Model line\u2014is always 1 in simple linear regression, and that the denominator degrees of freedom\u2014from the Error line\u2014is N\u00a0<\/span>\u2212\u00a02. In this instance, we have 18 people so our degrees of freedom for the denominator is 16. Going to our F\u00a0<\/span>table (a portion of which is shown in Table 13.1<\/span><\/a>), we find that the appropriate critical value for 1 and 16 degrees of freedom is F<\/span>*\u00a0= 4.49. (The complete F<\/span>\u00a0table can be found in Appendix\u00a0C<\/span><\/a>.)<\/p>\n\n

\n
\n

<\/a>Table 13.1.<\/span> Critical Values for F<\/span> (F<\/span> Table)<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

df:<\/span> Denominator (Within)<\/span><\/p>\n<\/td>\n

\n

df:<\/span> Numerator (Between)<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

1<\/span><\/p>\n<\/td>\n

\n

2<\/span><\/p>\n<\/td>\n

\n

3<\/span><\/p>\n<\/td>\n

\n

4<\/span><\/p>\n<\/td>\n

\n

5<\/span><\/p>\n<\/td>\n

\n

6<\/span><\/p>\n<\/td>\n

\n

7<\/span><\/p>\n<\/td>\n

\n

8<\/span><\/p>\n<\/td>\n

\n

9<\/span><\/p>\n<\/td>\n

\n

10<\/span><\/p>\n<\/td>\n

\n

11<\/span><\/p>\n<\/td>\n

\n

12<\/span><\/p>\n<\/td>\n

\n

14<\/span><\/p>\n<\/td>\n

\n

16<\/span><\/p>\n<\/td>\n

\n

20<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

1<\/span><\/p>\n<\/td>\n

\n

161<\/span><\/p>\n<\/td>\n

\n

200<\/span><\/p>\n<\/td>\n

\n

216<\/span><\/p>\n<\/td>\n

\n

225<\/span><\/p>\n<\/td>\n

\n

230<\/span><\/p>\n<\/td>\n

\n

234<\/span><\/p>\n<\/td>\n

\n

237<\/span><\/p>\n<\/td>\n

\n

239<\/span><\/p>\n<\/td>\n

\n

241<\/span><\/p>\n<\/td>\n

\n

242<\/span><\/p>\n<\/td>\n

\n

243<\/span><\/p>\n<\/td>\n

\n

244<\/span><\/p>\n<\/td>\n

\n

245<\/span><\/p>\n<\/td>\n

\n

246<\/span><\/p>\n<\/td>\n

\n

248<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

2<\/span><\/p>\n<\/td>\n

\n

18.51<\/span><\/p>\n<\/td>\n

\n

19.00<\/span><\/p>\n<\/td>\n

\n

19.16<\/span><\/p>\n<\/td>\n

\n

19.25<\/span><\/p>\n<\/td>\n

\n

19.30<\/span><\/p>\n<\/td>\n

\n

19.33<\/span><\/p>\n<\/td>\n

\n

19.36<\/span><\/p>\n<\/td>\n

\n

19.37<\/span><\/p>\n<\/td>\n

\n

19.38<\/span><\/p>\n<\/td>\n

\n

19.39<\/span><\/p>\n<\/td>\n

\n

19.40<\/span><\/p>\n<\/td>\n

\n

19.41<\/span><\/p>\n<\/td>\n

\n

19.42<\/span><\/p>\n<\/td>\n

\n

19.43<\/span><\/p>\n<\/td>\n

\n

19.44<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

3<\/span><\/p>\n<\/td>\n

\n

10.13<\/span><\/p>\n<\/td>\n

\n

9.55<\/span><\/p>\n<\/td>\n

\n

9.28<\/span><\/p>\n<\/td>\n

\n

9.12<\/span><\/p>\n<\/td>\n

\n

9.01<\/span><\/p>\n<\/td>\n

\n

8.94<\/span><\/p>\n<\/td>\n

\n

8.88<\/span><\/p>\n<\/td>\n

\n

8.84<\/span><\/p>\n<\/td>\n

\n

8.81<\/span><\/p>\n<\/td>\n

\n

8.78<\/span><\/p>\n<\/td>\n

\n

8.76<\/span><\/p>\n<\/td>\n

\n

8.74<\/span><\/p>\n<\/td>\n

\n

8.71<\/span><\/p>\n<\/td>\n

\n

8.69<\/span><\/p>\n<\/td>\n

\n

8.66<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

4<\/span><\/p>\n<\/td>\n

\n

7.71<\/span><\/p>\n<\/td>\n

\n

6.94<\/span><\/p>\n<\/td>\n

\n

6.59<\/span><\/p>\n<\/td>\n

\n

6.39<\/span><\/p>\n<\/td>\n

\n

6.26<\/span><\/p>\n<\/td>\n

\n

6.16<\/span><\/p>\n<\/td>\n

\n

6.09<\/span><\/p>\n<\/td>\n

\n

6.04<\/span><\/p>\n<\/td>\n

\n

6.00<\/span><\/p>\n<\/td>\n

\n

5.96<\/span><\/p>\n<\/td>\n

\n

5.93<\/span><\/p>\n<\/td>\n

\n

5.91<\/span><\/p>\n<\/td>\n

\n

5.87<\/span><\/p>\n<\/td>\n

\n

5.84<\/span><\/p>\n<\/td>\n

\n

5.80<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

5<\/span><\/p>\n<\/td>\n

\n

6.61<\/span><\/p>\n<\/td>\n

\n

5.79<\/span><\/p>\n<\/td>\n

\n

5.41<\/span><\/p>\n<\/td>\n

\n

5.19<\/span><\/p>\n<\/td>\n

\n

5.05<\/span><\/p>\n<\/td>\n

\n

4.95<\/span><\/p>\n<\/td>\n

\n

4.88<\/span><\/p>\n<\/td>\n

\n

4.82<\/span><\/p>\n<\/td>\n

\n

4.78<\/span><\/p>\n<\/td>\n

\n

4.74<\/span><\/p>\n<\/td>\n

\n

4.70<\/span><\/p>\n<\/td>\n

\n

4.68<\/span><\/p>\n<\/td>\n

\n

4.64<\/span><\/p>\n<\/td>\n

\n

4.60<\/span><\/p>\n<\/td>\n

\n

4.56<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

6<\/span><\/p>\n<\/td>\n

\n

5.99<\/span><\/p>\n<\/td>\n

\n

5.14<\/span><\/p>\n<\/td>\n

\n

4.76<\/span><\/p>\n<\/td>\n

\n

4.53<\/span><\/p>\n<\/td>\n

\n

4.39<\/span><\/p>\n<\/td>\n

\n

4.28<\/span><\/p>\n<\/td>\n

\n

4.21<\/span><\/p>\n<\/td>\n

\n

4.15<\/span><\/p>\n<\/td>\n

\n

4.10<\/span><\/p>\n<\/td>\n

\n

4.06<\/span><\/p>\n<\/td>\n

\n

4.03<\/span><\/p>\n<\/td>\n

\n

4.00<\/span><\/p>\n<\/td>\n

\n

3.96<\/span><\/p>\n<\/td>\n

\n

3.92<\/span><\/p>\n<\/td>\n

\n

3.87<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

7<\/span><\/p>\n<\/td>\n

\n

5.59<\/span><\/p>\n<\/td>\n

\n

4.74<\/span><\/p>\n<\/td>\n

\n

4.35<\/span><\/p>\n<\/td>\n

\n

4.12<\/span><\/p>\n<\/td>\n

\n

3.97<\/span><\/p>\n<\/td>\n

\n

3.87<\/span><\/p>\n<\/td>\n

\n

3.79<\/span><\/p>\n<\/td>\n

\n

3.73<\/span><\/p>\n<\/td>\n

\n

3.68<\/span><\/p>\n<\/td>\n

\n

3.63<\/span><\/p>\n<\/td>\n

\n

3.60<\/span><\/p>\n<\/td>\n

\n

3.57<\/span><\/p>\n<\/td>\n

\n

3.52<\/span><\/p>\n<\/td>\n

\n

3.49<\/span><\/p>\n<\/td>\n

\n

3.44<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

8<\/span><\/p>\n<\/td>\n

\n

5.32<\/span><\/p>\n<\/td>\n

\n

4.46<\/span><\/p>\n<\/td>\n

\n

4.07<\/span><\/p>\n<\/td>\n

\n

3.84<\/span><\/p>\n<\/td>\n

\n

3.69<\/span><\/p>\n<\/td>\n

\n

3.58<\/span><\/p>\n<\/td>\n

\n

3.50<\/span><\/p>\n<\/td>\n

\n

3.44<\/span><\/p>\n<\/td>\n

\n

3.39<\/span><\/p>\n<\/td>\n

\n

3.34<\/span><\/p>\n<\/td>\n

\n

3.31<\/span><\/p>\n<\/td>\n

\n

3.28<\/span><\/p>\n<\/td>\n

\n

3.23<\/span><\/p>\n<\/td>\n

\n

3.20<\/span><\/p>\n<\/td>\n

\n

3.15<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

9<\/span><\/p>\n<\/td>\n

\n

5.12<\/span><\/p>\n<\/td>\n

\n

4.26<\/span><\/p>\n<\/td>\n

\n

3.86<\/span><\/p>\n<\/td>\n

\n

3.63<\/span><\/p>\n<\/td>\n

\n

3.48<\/span><\/p>\n<\/td>\n

\n

3.37<\/span><\/p>\n<\/td>\n

\n

3.29<\/span><\/p>\n<\/td>\n

\n

3.23<\/span><\/p>\n<\/td>\n

\n

3.18<\/span><\/p>\n<\/td>\n

\n

3.13<\/span><\/p>\n<\/td>\n

\n

3.10<\/span><\/p>\n<\/td>\n

\n

3.07<\/span><\/p>\n<\/td>\n

\n

3.02<\/span><\/p>\n<\/td>\n

\n

2.98<\/span><\/p>\n<\/td>\n

\n

2.93<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

10<\/span><\/p>\n<\/td>\n

\n

4.96<\/span><\/p>\n<\/td>\n

\n

4.10<\/span><\/p>\n<\/td>\n

\n

3.71<\/span><\/p>\n<\/td>\n

\n

3.48<\/span><\/p>\n<\/td>\n

\n

3.33<\/span><\/p>\n<\/td>\n

\n

3.22<\/span><\/p>\n<\/td>\n

\n

3.14<\/span><\/p>\n<\/td>\n

\n

3.07<\/span><\/p>\n<\/td>\n

\n

3.02<\/span><\/p>\n<\/td>\n

\n

2.97<\/span><\/p>\n<\/td>\n

\n

2.94<\/span><\/p>\n<\/td>\n

\n

2.91<\/span><\/p>\n<\/td>\n

\n

2.86<\/span><\/p>\n<\/td>\n

\n

2.82<\/span><\/p>\n<\/td>\n

\n

2.77<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

11<\/span><\/p>\n<\/td>\n

\n

4.84<\/span><\/p>\n<\/td>\n

\n

3.98<\/span><\/p>\n<\/td>\n

\n

3.59<\/span><\/p>\n<\/td>\n

\n

3.36<\/span><\/p>\n<\/td>\n

\n

3.20<\/span><\/p>\n<\/td>\n

\n

3.09<\/span><\/p>\n<\/td>\n

\n

3.01<\/span><\/p>\n<\/td>\n

\n

2.95<\/span><\/p>\n<\/td>\n

\n

2.90<\/span><\/p>\n<\/td>\n

\n

2.86<\/span><\/p>\n<\/td>\n

\n

2.82<\/span><\/p>\n<\/td>\n

\n

2.79<\/span><\/p>\n<\/td>\n

\n

2.74<\/span><\/p>\n<\/td>\n

\n

2.70<\/span><\/p>\n<\/td>\n

\n

2.65<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

12<\/span><\/p>\n<\/td>\n

\n

4.75<\/span><\/p>\n<\/td>\n

\n

3.88<\/span><\/p>\n<\/td>\n

\n

3.49<\/span><\/p>\n<\/td>\n

\n

3.26<\/span><\/p>\n<\/td>\n

\n

3.11<\/span><\/p>\n<\/td>\n

\n

3.00<\/span><\/p>\n<\/td>\n

\n

2.92<\/span><\/p>\n<\/td>\n

\n

2.85<\/span><\/p>\n<\/td>\n

\n

2.80<\/span><\/p>\n<\/td>\n

\n

2.76<\/span><\/p>\n<\/td>\n

\n

2.72<\/span><\/p>\n<\/td>\n

\n

2.69<\/span><\/p>\n<\/td>\n

\n

2.64<\/span><\/p>\n<\/td>\n

\n

2.60<\/span><\/p>\n<\/td>\n

\n

2.54<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

13<\/span><\/p>\n<\/td>\n

\n

4.67<\/span><\/p>\n<\/td>\n

\n

3.80<\/span><\/p>\n<\/td>\n

\n

3.41<\/span><\/p>\n<\/td>\n

\n

3.18<\/span><\/p>\n<\/td>\n

\n

3.02<\/span><\/p>\n<\/td>\n

\n

2.92<\/span><\/p>\n<\/td>\n

\n

2.84<\/span><\/p>\n<\/td>\n

\n

2.77<\/span><\/p>\n<\/td>\n

\n

2.72<\/span><\/p>\n<\/td>\n

\n

2.67<\/span><\/p>\n<\/td>\n

\n

2.63<\/span><\/p>\n<\/td>\n

\n

2.60<\/span><\/p>\n<\/td>\n

\n

2.55<\/span><\/p>\n<\/td>\n

\n

2.51<\/span><\/p>\n<\/td>\n

\n

2.46<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

14<\/span><\/p>\n<\/td>\n

\n

4.60<\/span><\/p>\n<\/td>\n

\n

3.74<\/span><\/p>\n<\/td>\n

\n

3.34<\/span><\/p>\n<\/td>\n

\n

3.11<\/span><\/p>\n<\/td>\n

\n

2.96<\/span><\/p>\n<\/td>\n

\n

2.85<\/span><\/p>\n<\/td>\n

\n

2.77<\/span><\/p>\n<\/td>\n

\n

2.70<\/span><\/p>\n<\/td>\n

\n

2.65<\/span><\/p>\n<\/td>\n

\n

2.60<\/span><\/p>\n<\/td>\n

\n

2.56<\/span><\/p>\n<\/td>\n

\n

2.53<\/span><\/p>\n<\/td>\n

\n

2.48<\/span><\/p>\n<\/td>\n

\n

2.44<\/span><\/p>\n<\/td>\n

\n

2.39<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

15<\/span><\/p>\n<\/td>\n

\n

4.54<\/span><\/p>\n<\/td>\n

\n

3.68<\/span><\/p>\n<\/td>\n

\n

3.29<\/span><\/p>\n<\/td>\n

\n

3.06<\/span><\/p>\n<\/td>\n

\n

2.90<\/span><\/p>\n<\/td>\n

\n

2.79<\/span><\/p>\n<\/td>\n

\n

2.70<\/span><\/p>\n<\/td>\n

\n

2.64<\/span><\/p>\n<\/td>\n

\n

2.59<\/span><\/p>\n<\/td>\n

\n

2.55<\/span><\/p>\n<\/td>\n

\n

2.51<\/span><\/p>\n<\/td>\n

\n

2.48<\/span><\/p>\n<\/td>\n

\n

2.43<\/span><\/p>\n<\/td>\n

\n

2.39<\/span><\/p>\n<\/td>\n

\n

2.33<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

16<\/span><\/p>\n<\/td>\n

\n

4.49<\/span><\/p>\n<\/td>\n

\n

3.63<\/span><\/p>\n<\/td>\n

\n

3.24<\/span><\/p>\n<\/td>\n

\n

3.01<\/span><\/p>\n<\/td>\n

\n

2.85<\/span><\/p>\n<\/td>\n

\n

2.74<\/span><\/p>\n<\/td>\n

\n

2.66<\/span><\/p>\n<\/td>\n

\n

2.59<\/span><\/p>\n<\/td>\n

\n

2.54<\/span><\/p>\n<\/td>\n

\n

2.49<\/span><\/p>\n<\/td>\n

\n

2.45<\/span><\/p>\n<\/td>\n

\n

2.42<\/span><\/p>\n<\/td>\n

\n

2.37<\/span><\/p>\n<\/td>\n

\n

2.33<\/span><\/p>\n<\/td>\n

\n

2.28<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n

Step 3:<\/span> Calculate the Test Statistic and Effect Size<\/h5>\n

The process of calculating the test statistic for regression first involves computing the parameter estimates for the line of best fit. To do this, we first calculate the means, standard deviations, and sum of products for our X<\/span> and Y<\/span> variables, as shown below.<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

X<\/span><\/p>\n<\/td>\n

\n

(X<\/span> \u2212 M<\/span>X<\/span>)<\/p>\n<\/td>\n

\n

(X<\/span> \u2212 M<\/span>X<\/span>)2<\/span><\/p>\n<\/td>\n

\n

Y<\/span><\/p>\n<\/td>\n

\n

(Y<\/span> \u2212 M<\/span>Y<\/span>)<\/p>\n<\/td>\n

\n

(Y<\/span> \u2212 M<\/span>Y<\/span>)2<\/span><\/p>\n<\/td>\n

\n

(X<\/span> \u2212 M<\/span>X<\/span>)(Y<\/span> \u2212 M<\/span>Y<\/span>)<\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

17.65<\/p>\n<\/td>\n

\n

\u22122.13<\/p>\n<\/td>\n

\n

4.53<\/p>\n<\/td>\n

\n

10.36<\/p>\n<\/td>\n

\n

\u22127.10<\/p>\n<\/td>\n

\n

50.37<\/p>\n<\/td>\n

\n

15.10<\/p>\n<\/td>\n<\/tr>\n

\n

16.99<\/p>\n<\/td>\n

\n

\u22122.79<\/p>\n<\/td>\n

\n

7.80<\/p>\n<\/td>\n

\n

16.38<\/p>\n<\/td>\n

\n

\u22121.08<\/p>\n<\/td>\n

\n

1.16<\/p>\n<\/td>\n

\n

3.01<\/p>\n<\/td>\n<\/tr>\n

\n

18.30<\/p>\n<\/td>\n

\n

\u22121.48<\/p>\n<\/td>\n

\n

2.18<\/p>\n<\/td>\n

\n

15.23<\/p>\n<\/td>\n

\n

\u22122.23<\/p>\n<\/td>\n

\n

4.97<\/p>\n<\/td>\n

\n

3.29<\/p>\n<\/td>\n<\/tr>\n

\n

18.28<\/p>\n<\/td>\n

\n

\u22121.50<\/p>\n<\/td>\n

\n

2.25<\/p>\n<\/td>\n

\n

14.26<\/p>\n<\/td>\n

\n

\u22123.19<\/p>\n<\/td>\n

\n

10.18<\/p>\n<\/td>\n

\n

4.79<\/p>\n<\/td>\n<\/tr>\n

\n

21.89<\/p>\n<\/td>\n

\n

2.11<\/p>\n<\/td>\n

\n

4.47<\/p>\n<\/td>\n

\n

17.71<\/p>\n<\/td>\n

\n

0.26<\/p>\n<\/td>\n

\n

0.07<\/p>\n<\/td>\n

\n

0.55<\/p>\n<\/td>\n<\/tr>\n

\n

22.61<\/p>\n<\/td>\n

\n

2.83<\/p>\n<\/td>\n

\n

8.01<\/p>\n<\/td>\n

\n

16.47<\/p>\n<\/td>\n

\n

\u22120.98<\/p>\n<\/td>\n

\n

0.97<\/p>\n<\/td>\n

\n

\u22122.79<\/p>\n<\/td>\n<\/tr>\n

\n

17.42<\/p>\n<\/td>\n

\n

\u22122.36<\/p>\n<\/td>\n

\n

5.57<\/p>\n<\/td>\n

\n

16.89<\/p>\n<\/td>\n

\n

\u22120.56<\/p>\n<\/td>\n

\n

0.32<\/p>\n<\/td>\n

\n

1.33<\/p>\n<\/td>\n<\/tr>\n

\n

20.35<\/p>\n<\/td>\n

\n

0.57<\/p>\n<\/td>\n

\n

0.32<\/p>\n<\/td>\n

\n

18.74<\/p>\n<\/td>\n

\n

1.29<\/p>\n<\/td>\n

\n

1.66<\/p>\n<\/td>\n

\n

0.73<\/p>\n<\/td>\n<\/tr>\n

\n

18.89<\/p>\n<\/td>\n

\n

\u22120.89<\/p>\n<\/td>\n

\n

0.79<\/p>\n<\/td>\n

\n

21.96<\/p>\n<\/td>\n

\n

4.50<\/p>\n<\/td>\n

\n

20.26<\/p>\n<\/td>\n

\n

\u22124.00<\/p>\n<\/td>\n<\/tr>\n

\n

18.63<\/p>\n<\/td>\n

\n

\u22121.15<\/p>\n<\/td>\n

\n

1.32<\/p>\n<\/td>\n

\n

17.57<\/p>\n<\/td>\n

\n

0.11<\/p>\n<\/td>\n

\n

0.01<\/p>\n<\/td>\n

\n

\u22120.13<\/p>\n<\/td>\n<\/tr>\n

\n

19.67<\/p>\n<\/td>\n

\n

\u22120.11<\/p>\n<\/td>\n

\n

0.01<\/p>\n<\/td>\n

\n

18.12<\/p>\n<\/td>\n

\n

0.66<\/p>\n<\/td>\n

\n

0.44<\/p>\n<\/td>\n

\n

\u22120.08<\/p>\n<\/td>\n<\/tr>\n

\n

18.39<\/p>\n<\/td>\n

\n

\u22121.39<\/p>\n<\/td>\n

\n

1.94<\/p>\n<\/td>\n

\n

12.08<\/p>\n<\/td>\n

\n

\u22125.37<\/p>\n<\/td>\n

\n

28.87<\/p>\n<\/td>\n

\n

7.48<\/p>\n<\/td>\n<\/tr>\n

\n

22.48<\/p>\n<\/td>\n

\n

2.71<\/p>\n<\/td>\n

\n

7.32<\/p>\n<\/td>\n

\n

17.11<\/p>\n<\/td>\n

\n

\u22120.34<\/p>\n<\/td>\n

\n

0.12<\/p>\n<\/td>\n

\n

\u22120.93<\/p>\n<\/td>\n<\/tr>\n

\n

23.25<\/p>\n<\/td>\n

\n

3.47<\/p>\n<\/td>\n

\n

12.07<\/p>\n<\/td>\n

\n

21.66<\/p>\n<\/td>\n

\n

4.21<\/p>\n<\/td>\n

\n

17.73<\/p>\n<\/td>\n

\n

14.63<\/p>\n<\/td>\n<\/tr>\n

\n

19.91<\/p>\n<\/td>\n

\n

0.13<\/p>\n<\/td>\n

\n

0.02<\/p>\n<\/td>\n

\n

17.86<\/p>\n<\/td>\n

\n

0.40<\/p>\n<\/td>\n

\n

0.16<\/p>\n<\/td>\n

\n

0.05<\/p>\n<\/td>\n<\/tr>\n

\n

18.21<\/p>\n<\/td>\n

\n

\u22121.57<\/p>\n<\/td>\n

\n

2.45<\/p>\n<\/td>\n

\n

18.49<\/p>\n<\/td>\n

\n

1.03<\/p>\n<\/td>\n

\n

1.07<\/p>\n<\/td>\n

\n

\u22121.62<\/p>\n<\/td>\n<\/tr>\n

\n

23.65<\/p>\n<\/td>\n

\n

3.87<\/p>\n<\/td>\n

\n

14.99<\/p>\n<\/td>\n

\n

22.13<\/p>\n<\/td>\n

\n

4.67<\/p>\n<\/td>\n

\n

21.82<\/p>\n<\/td>\n

\n

18.08<\/p>\n<\/td>\n<\/tr>\n

\n

19.45<\/p>\n<\/td>\n

\n

\u22120.33<\/p>\n<\/td>\n

\n

0.11<\/p>\n<\/td>\n

\n

21.17<\/p>\n<\/td>\n

\n

3.72<\/p>\n<\/td>\n

\n

13.82<\/p>\n<\/td>\n

\n

\u22121.22<\/p>\n<\/td>\n<\/tr>\n

\n

356.02<\/p>\n<\/td>\n

\n

0.00<\/p>\n<\/td>\n

\n

76.14<\/p>\n<\/td>\n

\n

314.18<\/p>\n<\/td>\n

\n

0.00<\/p>\n<\/td>\n

\n

173.99<\/p>\n<\/td>\n

\n

58.29<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

From the raw data in our X<\/span> and Y<\/span> columns, we find that the means are \"\" = 19.78 and \"\" = 17.45. The deviation scores for each variable sum to zero, so all is well there. The sums of squares for X<\/span> and Y<\/span> ultimately lead us to standard deviations of s<\/span>X<\/span> = 2.12 and s<\/span>Y<\/span> = 3.20. Finally, our sum of products is 58.29, which gives us a covariance of covXY<\/span> = 3.43, so we know our relationship will be positive. This is all the information we need for our equations for the line of best fit.<\/p>\n

First, we must calculate the slope of the line:<\/p>\n

\"\"<\/p>\n

This means that as X<\/span> changes by 1 unit, Y<\/span> will change by 0.77. In terms of our problem, as health increases by 1, happiness goes up by 0.77, which is a positive relationship. Next, we use the slope, along with the means of each variable, to compute the intercept:<\/p>\n

\"\"<\/p>\n

For this particular problem (and most regressions), the intercept is not an important or interpretable value, so we will not read into it further. Now that we have all of our parameters estimated, we can give the full equation for our line of best fit:<\/p>\n

Y<\/span>\u0302<\/span> = 2.42 + 0.77X<\/span><\/p>\n

We can plot this relationship in a scatter plot and overlay our line onto it, as shown in Figure 13.3<\/span><\/a>.<\/p>\n\n

\n
\n

<\/a>Figure 13.3.<\/span> Health and happiness data and line of best fit. (\u201c<\/span>Scatter Plot Health and Happiness<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n\n<\/div>\n<\/div>\n

\n
\"\"<\/div>\n<\/div>\n

We can use the line equation to find predicted values for each observation and use them to calculate our sums of squares model and error, but this is tedious to do by hand, so we will let the computer software do the heavy lifting in that column of our ANOVA table:<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\n

Error<\/p>\n<\/td>\n

\n

129.37<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Now that we have these, we can fill in the rest of the ANOVA table. We already found our degrees of freedom in Step 2:<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n
\n

Error<\/p>\n<\/td>\n

\n

129.37<\/p>\n<\/td>\n

\n

16<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Our total line is always the sum of the other two lines, giving us:<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n
\n

Error<\/p>\n<\/td>\n

\n

129.37<\/p>\n<\/td>\n

\n

16<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

\n

173.99<\/p>\n<\/td>\n

\n

17<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Our mean squares column is only calculated for the model and error lines and is always our SS<\/span> divided by our d<\/span>f<\/span>, which is:<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
\n

Error<\/p>\n<\/td>\n

\n

129.37<\/p>\n<\/td>\n

\n

16<\/p>\n<\/td>\n

\n

8.09<\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

\n

173.99<\/p>\n<\/td>\n

\n

17<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Finally, our F\u00a0<\/span>statistic is the ratio of the mean squares:<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n
\n

Source<\/p>\n<\/td>\n

\n

SS<\/span><\/p>\n<\/td>\n

\n

df<\/span><\/p>\n<\/td>\n

\n

MS<\/span><\/p>\n<\/td>\n

\n

F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

\n

Model<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

44.62<\/p>\n<\/td>\n

\n

5.52<\/p>\n<\/td>\n<\/tr>\n

\n

Error<\/p>\n<\/td>\n

\n

129.37<\/p>\n<\/td>\n

\n

16<\/p>\n<\/td>\n

\n

8.09<\/p>\n<\/td>\n

<\/td>\n<\/tr>\n
\n

Total<\/p>\n<\/td>\n

\n

173.99<\/p>\n<\/td>\n

\n

17<\/p>\n<\/td>\n

<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

This gives us an obtained F\u00a0<\/span>statistic of 5.52, which we will use to test our hypothesis.<\/p>\n\n

Effect Size in Regression<\/h6>\n

We know that, because our statistic is significant, we should calculate an effect size. In regression, our effect size is variance explained, just like it was in ANOVA. Instead of using h<\/span>2<\/span> to represent this, we instead use R<\/span>2<\/span>, as we saw in correlation\u2014yet more evidence that all of these are the same analysis. (Note that in regression analysis, R<\/span>2<\/span> is typically capitalized, although for simple linear regression it represents the same value as r<\/span>2<\/span> we used in correlation.) Variance explained is still the ratio of SS<\/span>M<\/span> to SS<\/span>T<\/span>:<\/p>\n

\"\"<\/p>\n

We are explaining 26% of the variance in happiness based on health, which is a large effect size. (R<\/span>2<\/span>\u00a0uses the same effect size cutoffs as h<\/span>2<\/span>.)<\/p>\n\n

Step 4:<\/span> Make the Decision<\/h5>\n

We now have everything we need to make our final decision. Our obtained test statistic was F <\/span>= 5.52 and our critical value was F<\/span>* = 4.49. Since our obtained test statistic is greater than our critical value, we can reject the null hypothesis.<\/p>\n

Reject H<\/span>0<\/span>. Based on our sample of 18 people, we can predict levels of happiness based on how healthy someone is, and the effect size was large, F<\/span>(1, 16) = 5.52, p<\/span> < .05, R<\/span>2<\/span> < .26.<\/p>\n

Figure 13.4<\/span><\/a> shows the output from JASP for this example.<\/p>\n\n

\n
\n

<\/a>Figure 13.4.<\/span> Output from JASP for the linear regression described in the Happiness and Well-Being example. The output provides the slope (b<\/span>) of the line in the Coefficients table under Health Unstandardized (0.765), the y<\/span>-intercept (a<\/span>) of the line in the Coefficients table under (Intercept) Unstandardized (2.317, note slightly different from hand calculations due to rounding). The Model Summary \u2013 Happiness table provides R<\/span>2<\/span> (.256). The ANOVA table provides the result of the F<\/span> test statistic. Based on our sample of 18 people, we can predict levels of happiness based on how healthy someone is, F<\/span>(1, 16) = 5.5, p<\/span> = .032. (\u201c<\/span>JASP linear regression<\/span><\/span><\/a>\u201d by Rupa G. Gordon\/Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n\n<\/div>\n<\/div>\n

\n
\"\"<\/div>\n<\/div>\n

Accuracy in Prediction<\/h3>\n

We found a large, statistically significant relationship between our variables, which is what we hoped for. However, if we want to use our estimated line of best fit for future prediction, we will also want to know how precise or accurate our predicted values are. What we want to know is the average distance from our predictions to our actual observed values, or the average size of the residual (Y<\/span> \u2212 Y<\/span>\u0302<\/span>). The average size of the residual is known by a specific name: the [pb_glossary id=\"720\"]<\/a>[\/pb_glossary]standard error of the estimate<\/span> (\"\"), which is given by the formula<\/p>\n

\"\"<\/p>\n

This formula is almost identical to our standard deviation formula, and it follows the same logic. We square our residuals, add them up, then divide by the degrees of freedom. Although this sounds like a long process, we already have the sum of the squared residuals in our ANOVA table! In fact, the value under the square root sign is just the SS<\/span>E<\/span> divided by the d<\/span>f<\/span>E<\/span>, which is called the [pb_glossary id=\"716\"]<\/a>[\/pb_glossary]mean squared error<\/span>, or MS<\/span>E<\/span>:<\/p>\n

\"\"<\/p>\n

For our example:<\/p>\n

\"\"<\/p>\n

So on average, our predictions are just under 3 points away from our actual values. There are no specific cutoffs or guidelines for how big our standard error of the estimate can or should be; it is highly dependent on both our sample size and the scale of our original Y<\/span> variable, so expert judgment should be used. In this case, the estimate is not that far off and can be considered reasonably precise.<\/p>\n\n

Multiple Regression and Other Extensions<\/h3>\n

Simple linear regression as presented here is only a stepping stone toward an entire field of research and application. Regression is an incredibly flexible and powerful tool, and the extensions and variations on it are far beyond the scope of this chapter. (Indeed, even entire books struggle to accommodate all possible applications of the simple principles laid out here.) The next step in regression is to study [pb_glossary id=\"718\"]<\/a>[\/pb_glossary]multiple regression<\/span>, which uses multiple X<\/span> variables as predictors for a single Y<\/span>\u00a0variable at the same time. The math of multiple regression is very complex but the logic is the same: we are trying to use variables that are statistically significantly related to our outcome to explain the variance we observe in that outcome. Other forms of regression include [pb_glossary id=\"712\"]<\/a>[\/pb_glossary]curvilinear models<\/span> that can explain curves in the data rather than the straight lines used here, as well as [pb_glossary id=\"717\"]<\/a>[\/pb_glossary]moderation models<\/span> that change the relationship between two variables based on levels of a third. The possibilities are truly endless and offer a lifetime of discovery.<\/p>\n\n

Exercises<\/h3>\n
    \n \t
  1. How are ANOVA and linear regression similar? How are they different?<\/li>\n \t
  2. What is a residual?<\/li>\n \t
  3. How are correlation and regression similar? How are they different?<\/li>\n \t
  4. What are the two parameters of the line of best fit, and what do they represent?<\/li>\n \t
  5. What is our criterion for finding the line of best fit?<\/li>\n \t
  6. Fill out the rest of the ANOVA tables below for simple linear regressions:\n
      \n \t
    1. \n<\/colgroup>\n\n\n\n\n\n\n
      \n

      Source<\/p>\n<\/td>\n

      		\n

      SS<\/span><\/p>\n<\/td>\n

      		\n

      df<\/span><\/p>\n<\/td>\n

      		\n

      MS<\/span><\/p>\n<\/td>\n

      		\n

      F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

      \n

      Model<\/p>\n<\/td>\n

      		\n

      34.21<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
      \n

      Error<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
      \n

      Total<\/p>\n<\/td>\n

      		\n

      66.12<\/p>\n<\/td>\n

      		\n

      54<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n \t
    2. \n<\/colgroup>\n\n\n\n\n\n\n
      \n

      Source<\/p>\n<\/td>\n

      		\n

      SS<\/span><\/p>\n<\/td>\n

      		\n

      df<\/span><\/p>\n<\/td>\n

      		\n

      MS<\/span><\/p>\n<\/td>\n

      		\n

      F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

      \n

      Model<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n\n

      6.03<\/p>\n<\/td>\n

      		<\/td>\n<\/tr>\n
      \n

      Error<\/p>\n<\/td>\n

      		<\/td>\n\n

      16<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n<\/tr>\n
      \n

      Total<\/p>\n<\/td>\n

      		\n

      19.98<\/p>\n<\/td>\n

      		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<\/ol>\n<\/li>\n \t
    3. In Chapter 12<\/span><\/a>, we found a statistically significant correlation between overall performance in class and how much time someone studied. Use the summary statistics calculated in that problem (\"\" = 1.61, s<\/span>X<\/span> = 1.12, \"\" = 2.95, s<\/span>Y<\/span> = 0.99, r <\/span>= .65) to compute a line of best fit predicting success from study times.<\/li>\n \t
    4. Using the line of best fit equation created in Problem 7, predict the scores for how successful people will be based on how much they study:\n
        \n \t
      1. X<\/span> = 1.20<\/li>\n \t
      2. X<\/span> = 3.33<\/li>\n \t
      3. X<\/span> = 0.71<\/li>\n \t
      4. X<\/span> = 4.00<\/li>\n<\/ol>\n<\/li>\n \t
      5. You have become suspicious that the draft rankings of your fantasy football league have no predictive value for how teams place at the end of the season. You go back to historical league data and find rankings of teams after the draft and at the end of the season (shown in the following table) to test for a statistically significant predictive relationship. Assume SS<\/span>M<\/span> = 2.65 and SS<\/span>E<\/span>\u00a0=\u00a0337.35.\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
        \n

        Draft Projection<\/p>\n<\/td>\n

        		\n

        Final Rankings<\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        1<\/p>\n<\/td>\n

        		\n

        14<\/p>\n<\/td>\n<\/tr>\n

        \n

        2<\/p>\n<\/td>\n

        		\n

        6<\/p>\n<\/td>\n<\/tr>\n

        \n

        3<\/p>\n<\/td>\n

        		\n

        8<\/p>\n<\/td>\n<\/tr>\n

        \n

        4<\/p>\n<\/td>\n

        		\n

        13<\/p>\n<\/td>\n<\/tr>\n

        \n

        5<\/p>\n<\/td>\n

        		\n

        2<\/p>\n<\/td>\n<\/tr>\n

        \n

        6<\/p>\n<\/td>\n

        		\n

        15<\/p>\n<\/td>\n<\/tr>\n

        \n

        7<\/p>\n<\/td>\n

        		\n

        4<\/p>\n<\/td>\n<\/tr>\n

        \n

        8<\/p>\n<\/td>\n

        		\n

        10<\/p>\n<\/td>\n<\/tr>\n

        \n

        9<\/p>\n<\/td>\n

        		\n

        11<\/p>\n<\/td>\n<\/tr>\n

        \n

        10<\/p>\n<\/td>\n

        		\n

        16<\/p>\n<\/td>\n<\/tr>\n

        \n

        11<\/p>\n<\/td>\n

        		\n

        9<\/p>\n<\/td>\n<\/tr>\n

        \n

        12<\/p>\n<\/td>\n

        		\n

        7<\/p>\n<\/td>\n<\/tr>\n

        \n

        13<\/p>\n<\/td>\n

        		\n

        14<\/p>\n<\/td>\n<\/tr>\n

        \n

        14<\/p>\n<\/td>\n

        		\n

        12<\/p>\n<\/td>\n<\/tr>\n

        \n

        15<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n<\/tr>\n

        \n

        16<\/p>\n<\/td>\n

        		\n

        5<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n \t

      6. You have summary data for two variables: how extroverted someone is (X<\/span>) and how often someone volunteers (Y<\/span>). Using these values, calculate the line of best fit predicting volunteering from extroversion, then test for a statistically significant relationship using the hypothesis-testing procedure: \"\" = 12.58, s<\/span>X<\/span> = 4.65, \"\" = 7.44, s<\/span>Y<\/span> = 2.12, r <\/span>= .34, N <\/span>= 67, SS<\/span>M<\/span> = 19.79, SS<\/span>E<\/span>\u00a0=\u00a0215.77.<\/li>\n<\/ol>\n
        \n

        Answers to Odd-Numbered Exercises<\/h3>\n<\/header>\n
        \n\n1)\nANOVA and simple linear regression both take the total observed variance and partition it into pieces that we can explain and cannot explain and use the ratio of those pieces to test for significant relationships. They are different in that ANOVA uses a categorical variable as a predictor, whereas linear regression uses a continuous variable.\n\n3)\nCorrelation and regression both involve taking two continuous variables and finding a linear relationship between them. Correlations find a standardized value describing the direction and magnitude of the relationship, whereas regression finds the line of best fit and uses it to partition and explain variance.<\/span>\n\n5)\nLeast squares error solution; the line that minimizes the total amount of residual error in the dataset<\/span>\n\n7)\nb<\/span> = <\/span>r<\/span>*(<\/span>s<\/span>Y<\/span>\/<\/span>s<\/span>X<\/span>) = .65*(0.99\/1.12) = .57; <\/span>a<\/span> = <\/span>\"\" \u2212 <\/span>b<\/span>\"\" = 2.95 \u2212 (0.57)(1.61) = 3.87; <\/span>Y\u00a0<\/span>=\u00a03.87\u00a0+\u00a00.57<\/span>X<\/span>\n\n9)\nStep 1:<\/span> H<\/span>0<\/span>: <\/span>b<\/span> = 0 \u201cThere is no predictive relationship between draft rankings and final rankings in fantasy football,\u201d <\/span>H<\/span>A<\/span>: <\/span>b<\/span> \u2260 0 \u201cThere is a predictive relationship between draft rankings and final rankings in fantasy football.\u201d\n<\/span>Step 2:<\/span> Our model will have 1 (based on the number of predictors) and 14 (based on how many observations we have) degrees of freedom, giving us a critical value of <\/span>F<\/span>* = 4.60.\n<\/span>Step 3:<\/span> Using the sum of products table, we find <\/span>\"\" = 8.50, <\/span>\"\" = 8.50, <\/span>SS<\/span>X<\/span> = 339.86, <\/span>SP<\/span> = 29.99, giving us a line of best fit of: <\/span>b<\/span> = 29.99\/339.86 = 0.09; <\/span>a<\/span> = 8.50 \u2212 (0.09)(8.50) = 7.74; <\/span>Y\u00a0<\/span>=\u00a07.74\u00a0+\u00a00.09<\/span>X<\/span>. Our given <\/span>SS<\/span> values and our <\/span>d<\/span>f<\/span> from Step 2 allow us to fill in the ANOVA table:<\/span>\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        2.65<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		\n

        2.65<\/p>\n<\/td>\n

        		\n

        0.11<\/p>\n<\/td>\n<\/tr>\n

        \n

        Error<\/p>\n<\/td>\n

        		\n

        337.35<\/p>\n<\/td>\n

        		\n

        14<\/p>\n<\/td>\n

        		\n

        24.10<\/p>\n<\/td>\n

        		<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		\n

        339.86<\/p>\n<\/td>\n

        		\n

        15<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nStep 4:<\/span> Our obtained value was smaller than our critical value, so we fail to reject the null hypothesis. There is no evidence to suggest that draft rankings have any predictive value for final fantasy football rankings, F<\/span>(1, 14) = 0.11, p<\/span> > .05\n\n<\/div>\n<\/div>\n ","rendered":"
        \n
        \n

        Key Terms<\/h2>\n<\/header>\n
        \n

        curvilinear models<\/span><\/a><\/p>\n

        error<\/span><\/a><\/p>\n

        least squares error solution<\/span><\/a><\/p>\n

        line of best fit<\/span><\/a><\/p>\n

        mean squared error<\/span><\/a><\/p>\n

        moderation models<\/span><\/a><\/p>\n

        multiple regression<\/span><\/a><\/p>\n

        residual<\/span><\/a><\/p>\n

        standard error of the estimate<\/span><\/a><\/p>\n<\/div>\n<\/div>\n

        In Chapter 11<\/span><\/a>, we learned about ANOVA, which involves a new way of looking at how our data are structured and the inferences we can draw from that. In Chapter 12<\/span><\/a>, we learned about correlations, which analyze two continuous variables at the same time to see if they systematically relate in a linear fashion. In this chapter, we will combine these two techniques in an analysis called simple linear regression, or regression for short. Regression uses the technique of variance partitioning from ANOVA to more formally assess the types of relationships looked at in correlations. Regression is the most general and most flexible analysis covered in this book, and we will only scratch the surface.<\/p>\n

        Line of Best Fit<\/h3>\n

        In correlations, we referred to a linear trend in the data. That is, we assumed that there was a straight line we could draw through the middle of our scatter plot that would represent the relationship between our two variables, X<\/span> and Y<\/span>. Regression involves solving for the equation of that line, which is called the line of best fit.<\/p>\n

        The <\/a><\/a>line of best fit<\/span> can be thought of as the central tendency of our scatter plot. The term best fit<\/span> means that the line is as close to all points (with each point representing both variables for a single person) in the scatter plot as possible, with a balance of scores above and below the line. This is the same idea as the mean, which has an equal weighting of scores above and below it and is the best singular descriptor of all our data points for a single variable.<\/p>\n

        We have already seen many scatter plots in Chapter 2<\/span><\/a> and Chapter 12<\/span><\/a>, so we know by now that no scatter plot has points that form a perfectly straight line. Because of this, when we put a straight line through a scatter plot, it will not touch all of the points, and it may not even touch any! This will result in some distance between the line and each of the points it is supposed to represent, just like a mean has some distance between it and all of the individual scores in the dataset.<\/p>\n

        The distances between the line of best fit and each individual data point go by two different names that mean the same thing: errors and residuals. The term <\/a><\/a>error<\/span> in regression is closely aligned with the meaning of error in statistics (think standard error or sampling error); it does not mean that we did anything wrong, it simply means that there was some discrepancy or difference between what our analysis produced and the true value we are trying to get at. The term <\/a><\/a>residual<\/span> is new to our study of statistics, and it takes on a very similar meaning in regression to what it means in everyday parlance: there is something left over. In regression, what is \u201cleft over\u201d\u2014that is, what makes up the residual\u2014is an imperfection in our ability to predict values of the Y<\/span> variable using our line. This definition brings us to one of the primary purposes of regression and the line of best fit: predicting\u00a0scores.<\/p>\n

        Prediction<\/h3>\n

        The goal of regression is the same as the goal of ANOVA: to take what we know about one variable (X<\/span>) and use it to explain our observed differences in another variable (Y<\/span>). In ANOVA, we talked about\u2014and tested for\u2014group mean differences, but in regression we do not have groups for our explanatory variable; we have a continuous variable, like in correlation. Because of this, our vocabulary will be a little bit different, but the process, logic, and end result are all the same.<\/p>\n

        In regression, we most frequently talk about prediction, specifically predicting our outcome variable Y<\/span> from our explanatory variable X<\/span>, and we use the line of best fit to make our predictions. Let\u2019s take a look at the equation for the line, which is quite simple:<\/p>\n

        Y\u0302<\/span> = a<\/span> + bX<\/span><\/p>\n

        The terms in the equation are defined as:<\/p>\n

        Y<\/span>\u0302<\/span>: the predicted value of Y<\/span> for an individual person<\/p>\n

        a<\/span>: the intercept of the line<\/p>\n

        b<\/span>: the slope of the line<\/p>\n

        X<\/span>: the observed value of X<\/span> for an individual person<\/p>\n

        What this shows us is that we will use our known value of X<\/span> for each person to predict the value of Y<\/span> for that person. The predicted value, Y\u0302<\/span> (\u201cY<\/span>-hat\u201d), is our best guess for what a person\u2019s score on the outcome is. Notice also that the form of the equation is very similar to very simple linear equations that you have likely encountered before and has only two parameter estimates: an intercept (where the line crosses the y<\/span>-axis) and a slope (how steep\u2014and the direction, positive or negative\u2014the line is). These are parameter estimates because, like everything else in statistics, we are interested in approximating the true value of the relationship in the population but can only ever estimate it using sample data. We will soon see that one of these parameters, the slope, is the focus of our hypothesis tests (the intercept is only there to make the math work out properly and is rarely interpretable). The formulas for these parameter estimates use very familiar values:<\/p>\n

        \"\"<\/p>\n

        <\/a>\"\"<\/p>\n

        We have seen each of these before. \"\" and \"\" are the means of Y<\/span> and X<\/span>, respectively; \"\" is the covariance of X<\/span> and Y<\/span> we learned about with correlations; and \"\" is the variance of X<\/span>. The formula for the slope is very similar to the formula for a Pearson correlation coefficient; the only difference is that we are dividing by the variance of X<\/span> instead of the product of the standard deviations of X<\/span> and Y<\/span>. Because of this, our slope is scaled to the same scale as our X<\/span> variable and is no longer constrained to be between 0 and 1 in absolute value. This formula provides a clear definition of the slope of the line of best fit, and just like with correlation, this definitional formula can be simplified into a short computational formula for easier calculations. In this case, we are simply taking the sum of products and dividing by the sum of squares for X<\/span>.<\/p>\n

        Notice that there is a third formula for the slope of the line that involves the correlation between X<\/span>\u00a0and Y<\/span>. This is because regression and correlation look for the same thing: a straight line through the middle of the data. The only difference between a regression coefficient in simple linear regression and a Pearson correlation coefficient is the scale. So, if you lack raw data but have summary information on the correlation and standard deviations for variables, you can still compute a slope, and therefore an intercept, for a line of best fit.<\/p>\n

        It is important to point out that the Y<\/span> values in the equations for a<\/span> and b<\/span> are our observed Y<\/span>\u00a0values in the dataset, not<\/span> the predicted Y<\/span> values (Y<\/span>\u0302<\/span>) from our equation for the line of best fit. Thus, we will have three values for each person: the observed value of X<\/span> (X<\/span>), the observed value of Y<\/span> (Y<\/span>), and the predicted value of Y<\/span> (Y<\/span>\u0302<\/span>). You may be asking why we would try to predict Y<\/span> if we have an observed value of Y<\/span>, and that is a reasonable question. The answer has two explanations. First, we need to use known values of Y<\/span> to calculate the parameter estimates in our equation, and we use the difference between our observed values and predicted values (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) to see how accurate our equation is. And second, we often use regression to create a predictive model that we can then use to predict values of Y<\/span> for other people for whom we only have information on X<\/span>.<\/p>\n

        Let\u2019s look at this from an applied example. Businesses often have more applicants for a job than they have openings available, so they want to know who among the applicants is most likely to be the best employee. There are many criteria that can be used, but one is a personality test for conscientiousness, with the belief being that more conscientious (more responsible) employees are better than less conscientious employees. A business might give their employees a personality inventory to assess conscientiousness and study existing performance data to look for a relationship. In this example, we have known values of the predictor (X<\/span>, conscientiousness) and outcome (Y<\/span>, job performance), so we can estimate an equation for a line of best fit and see how accurately conscientiousness predicts job performance, then use this equation to predict future job performance of applicants based only on their known values of conscientiousness from personality inventories given during the application process.<\/p>\n

        The key in assessing whether a linear regression works well is the difference between our observed and known Y<\/span> values and our predicted Y<\/span>\u0302<\/span> values. As mentioned in passing above, we use subtraction to find the difference between them (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) in the same way we use subtraction for deviation scores and sums of squares. The value (Y<\/span> \u2212 Y<\/span>\u0302<\/span>) is our residual, which, as defined above, is how close our line of best fit is to our actual values. We can visualize residuals to get a better sense of what they are by creating a scatter plot and overlaying a line of best fit on it, as shown in Figure 13.1<\/span><\/a>.<\/p>\n

        \n
        \n

        <\/a>Figure 13.1.<\/span> Scatter plot with residuals. (\u201c<\/span>Scatter Plot with Residuals<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n<\/div>\n<\/div>\n

        \n
        \"\"<\/div>\n<\/div>\n

        In Figure 13.1<\/span><\/a>, the triangular dots represent observations from each person on both X<\/span> and Y<\/span> and the dotted line is the line of best fit estimated by the equation Y<\/span>\u0302<\/span> = a<\/span> + bX<\/span>. For every person in the dataset, the line represents their predicted score. The brackets between the triangular dots and the predicted scores on the line of best fit are our residuals. (For ease of viewing, they are only drawn for four observations, but in reality there is one for every observation.) You can see that some residuals are positive and some are negative, and that some are very large and some are very small. This means that some predictions are very accurate and some are very inaccurate, and that some predictions overestimate values and some underestimate values. Across the entire dataset, the line of best fit is the one that minimizes the total (sum) value of all residuals. That is, although predictions at an individual level might be somewhat inaccurate, across our full sample and (theoretically) in future samples our total amount of error is as small as possible. We call this property of the line of best fit the <\/a><\/a>least squares error solution<\/span>. This term means that the solution\u2014or equation\u2014of the line is the one that provides the smallest possible value of the squared errors (squared so that they can be summed, just like in standard deviation) relative to any other straight line we could draw through the data.<\/p>\n

        Predicting Scores and Explaining Variance<\/h4>\n

        We have now seen that the purpose of regression is twofold: we want to predict scores based on our line and, as stated earlier, explain variance in our observed Y<\/span> variable just like in ANOVA. These two purposes go hand in hand, and our ability to predict scores is literally our ability to explain variance. That is, if we cannot account for the variance in Y<\/span> based on X<\/span>, then we have no reason to use X<\/span> to predict future values of Y<\/span>.<\/p>\n

        We know that the overall variance in Y<\/span> is a function of each score deviating from the mean of Y<\/span> (as in our calculation of variance and standard deviation). So, just like the brackets in Figure 13.1<\/span><\/a> representing residuals, given as (Y<\/span> \u2212 Y<\/span>\u0302<\/span>), we can visualize the overall variance as each score\u2019s distance from the overall mean of Y<\/span>, given as (\"\"), our normal deviation score. This is shown in Figure 13.2<\/span><\/a>.<\/p>\n

        \n
        \n

        <\/a>Figure 13.2.<\/span> Scatter plot with residuals and deviation scores. (\u201c<\/span>Scatter Plot with Residuals and Deviation Scores<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n<\/div>\n<\/div>\n

        \n
        \"\"<\/div>\n<\/div>\n

        In Figure 13.2<\/span><\/a>, the solid line is the mean of Y<\/span>, and the blue brackets are the deviation scores between our observed values of Y<\/span> and the mean of Y<\/span>. This represents the overall variance that we are trying to explain. Thus, the residuals and the deviation scores are the same type of idea: the distance between an observed score and a given line, either the line of best fit that gives predictions or the line representing the mean that serves as a baseline. The difference between these two values, which is the distance between the lines themselves, is our model\u2019s ability to predict scores above and beyond the baseline mean; that is, it is our model\u2019s ability to explain the variance we observe in Y<\/span> based on values of X<\/span>. If we have no ability to explain variance, then our line will be flat (the slope will be 0.00) and will be the same as the line representing the mean, and the distance between the lines will be 0.00 as well.<\/p>\n

        We now have three pieces of information: the distance from the observed score to the mean, the distance from the observed score to the prediction line, and the distance from the prediction line to the mean. These are our three pieces of information needed to test our hypotheses about regression and to calculate effect sizes. They are our three sums of squares, just like in ANOVA. Our distance from the observed score to the mean is the sum of squares total, which we are trying to explain. Our distance from the observed score to the prediction line is our sum of squares error, or residual, which we are trying to minimize. Our distance from the prediction line to the mean is our sum of squares model, which is our observed effect and our ability to explain variance. Each of these will go into the ANOVA table to calculate our test statistic.<\/p>\n

        ANOVA Table<\/h3>\n

        Our ANOVA table in regression follows the exact same format as it did for ANOVA (hence the name). The top row (Model) is our observed effect, the middle row is our error, and the bottom row is our total. The columns take on the same interpretations as well: from left to right, we have our sums of squares, our degrees of freedom, our mean squares, and our F\u00a0<\/span>statistic.<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        \u2211<\/span>(Y<\/span>\u0302<\/span> \u2212 \"\"<\/span>)2<\/span><\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		\n

        \"\"<\/span><\/p>\n<\/td>\n

        		\n

        \"\"<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        Error<\/p>\n<\/td>\n

        		\n

        \u2211<\/span>(Y<\/span> \u2212 Y<\/span>\u0302<\/span>)2<\/span><\/p>\n<\/td>\n

        		\n

        N <\/span>\u2212 2<\/p>\n<\/td>\n

        		\n

        \"\"<\/span><\/p>\n<\/td>\n

        		<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		\n

        \u2211<\/span>(\"\"<\/span>)2<\/span><\/p>\n<\/td>\n

        		\n

        N <\/span>\u2212 1<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        As with ANOVA, getting the values for the SS<\/span> column is a straightforward but somewhat arduous process. First, you take the raw scores of X<\/span> and Y<\/span> and calculate the means, variances, and covariance using the sum of products table introduced in our chapter on correlations<\/span><\/a>. Next, you use the variance of X<\/span> and the covariance of X<\/span> and Y<\/span> to calculate the slope of the line, b<\/span>. (The formula for calculating <\/span>b<\/span><\/a> was provided earlier.) After that, you use the means and the slope to find the intercept, a<\/span>, which is given alongside b<\/span>. After that, you use the full prediction equation for the line of best fit to get predicted Y<\/span> scores (Y<\/span>\u0302<\/span>) for each person. Finally, you use the observed Y<\/span> scores, predicted Y<\/span> scores, and mean of Y<\/span> (\"\") to find the appropriate deviation scores for each person for each sum of squares source in the table and sum them to get the sum of squares model, sum of squares error, and sum of squares total. As with ANOVA, you won\u2019t be required to compute the SS<\/span> values by hand, but you will need to know what they represent and how they fit together.<\/p>\n

        The other columns in the ANOVA table are all familiar. The degrees of freedom column still has N <\/span>\u2212 1 for our total, but now we have N <\/span>\u2212 2 for our error degrees of freedom and 1 for our model degrees of freedom; this is because simple linear regression only has one predictor, so our degrees of freedom for the model is always 1 and does not change. The total degrees of freedom must still be the sum of the other two, so our degrees of freedom error will always be N <\/span>\u2212 2 for simple linear regression. The mean square columns are still the SS<\/span> column divided by the d<\/span>f<\/span> column, and the test statistic F <\/span>is still the ratio of the mean squares. Based on this, it is now explicitly clear that not only do regression and ANOVA have the same goal but they are, in fact, the same analysis entirely. The only difference is the type of data we feed into the predictor side of the equations: continuous for regression and categorical for ANOVA.<\/p>\n

        Hypothesis Testing in Regression<\/h3>\n

        Regression, like all other analyses, will test a null hypothesis in our data. In regression, we are interested in predicting Y<\/span> scores and explaining variance using a line, the slope of which is what allows us to get closer to our observed scores than the mean of Y<\/span> can. Thus, our hypotheses concern the slope of the line, which is estimated in the prediction equation by b<\/span> (the slope of a population, as opposed to b<\/span>, which is the slope of a sample). Specifically, we want to test that the slope is not zero:<\/p>\n

        \"\"<\/p>\n

        \"\"<\/p>\n

        A non-zero slope indicates that we can explain values in Y<\/span> based on X<\/span> and therefore predict future values of Y<\/span> based on X<\/span>. Our alternative hypotheses are analogous to those in correlation: positive relationships have values above zero, negative relationships have values below zero, and two-tailed tests are possible. Just like ANOVA, we will test the significance of this relationship using the F\u00a0<\/span>statistic calculated in our ANOVA table compared to a critical value from the F\u00a0<\/span>distribution table. Let\u2019s take a look at an example and regression in action.<\/p>\n

        Example <\/span> Happiness and Well-being<\/p>\n

        Researchers are interested in explaining differences in how happy people are based on how healthy people are. They gather data on each of these variables from 18 people and fit a linear regression model to explain the variance. We will follow the four-step hypothesis-testing procedure to see if there is a relationship between these variables that is statistically significant.<\/p>\n

        Step 1:<\/span> State the Hypotheses<\/h5>\n

        The null hypothesis in regression states that there is no relationship between our variables. The alternative states that there is a relationship, but because our research description did not explicitly state a direction of the relationship, we will use a non-directional hypothesis.<\/p>\n

        \"\"<\/p>\n

        \"\"<\/p>\n

        Step 2:<\/span> Find the Critical Value<\/h5>\n

        Because regression and ANOVA are the same analysis, our critical value for regression will come from the same place: the F\u00a0<\/span>distribution table, which uses two types of degrees of freedom. We saw in the ANOVA table<\/span><\/a> that the degrees of freedom for our numerator\u2014the Model line\u2014is always 1 in simple linear regression, and that the denominator degrees of freedom\u2014from the Error line\u2014is N\u00a0<\/span>\u2212\u00a02. In this instance, we have 18 people so our degrees of freedom for the denominator is 16. Going to our F\u00a0<\/span>table (a portion of which is shown in Table 13.1<\/span><\/a>), we find that the appropriate critical value for 1 and 16 degrees of freedom is F<\/span>*\u00a0= 4.49. (The complete F<\/span>\u00a0table can be found in Appendix\u00a0C<\/span><\/a>.)<\/p>\n

        \n
        \n

        <\/a>Table 13.1.<\/span> Critical Values for F<\/span> (F<\/span> Table)<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
        \n

        df:<\/span> Denominator (Within)<\/span><\/p>\n<\/td>\n

        		\n

        df:<\/span> Numerator (Between)<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        1<\/span><\/p>\n<\/td>\n

        		\n

        2<\/span><\/p>\n<\/td>\n

        		\n

        3<\/span><\/p>\n<\/td>\n

        		\n

        4<\/span><\/p>\n<\/td>\n

        		\n

        5<\/span><\/p>\n<\/td>\n

        		\n

        6<\/span><\/p>\n<\/td>\n

        		\n

        7<\/span><\/p>\n<\/td>\n

        		\n

        8<\/span><\/p>\n<\/td>\n

        		\n

        9<\/span><\/p>\n<\/td>\n

        		\n

        10<\/span><\/p>\n<\/td>\n

        		\n

        11<\/span><\/p>\n<\/td>\n

        		\n

        12<\/span><\/p>\n<\/td>\n

        		\n

        14<\/span><\/p>\n<\/td>\n

        		\n

        16<\/span><\/p>\n<\/td>\n

        		\n

        20<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        1<\/span><\/p>\n<\/td>\n

        		\n

        161<\/span><\/p>\n<\/td>\n

        		\n

        200<\/span><\/p>\n<\/td>\n

        		\n

        216<\/span><\/p>\n<\/td>\n

        		\n

        225<\/span><\/p>\n<\/td>\n

        		\n

        230<\/span><\/p>\n<\/td>\n

        		\n

        234<\/span><\/p>\n<\/td>\n

        		\n

        237<\/span><\/p>\n<\/td>\n

        		\n

        239<\/span><\/p>\n<\/td>\n

        		\n

        241<\/span><\/p>\n<\/td>\n

        		\n

        242<\/span><\/p>\n<\/td>\n

        		\n

        243<\/span><\/p>\n<\/td>\n

        		\n

        244<\/span><\/p>\n<\/td>\n

        		\n

        245<\/span><\/p>\n<\/td>\n

        		\n

        246<\/span><\/p>\n<\/td>\n

        		\n

        248<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        2<\/span><\/p>\n<\/td>\n

        		\n

        18.51<\/span><\/p>\n<\/td>\n

        		\n

        19.00<\/span><\/p>\n<\/td>\n

        		\n

        19.16<\/span><\/p>\n<\/td>\n

        		\n

        19.25<\/span><\/p>\n<\/td>\n

        		\n

        19.30<\/span><\/p>\n<\/td>\n

        		\n

        19.33<\/span><\/p>\n<\/td>\n

        		\n

        19.36<\/span><\/p>\n<\/td>\n

        		\n

        19.37<\/span><\/p>\n<\/td>\n

        		\n

        19.38<\/span><\/p>\n<\/td>\n

        		\n

        19.39<\/span><\/p>\n<\/td>\n

        		\n

        19.40<\/span><\/p>\n<\/td>\n

        		\n

        19.41<\/span><\/p>\n<\/td>\n

        		\n

        19.42<\/span><\/p>\n<\/td>\n

        		\n

        19.43<\/span><\/p>\n<\/td>\n

        		\n

        19.44<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        3<\/span><\/p>\n<\/td>\n

        		\n

        10.13<\/span><\/p>\n<\/td>\n

        		\n

        9.55<\/span><\/p>\n<\/td>\n

        		\n

        9.28<\/span><\/p>\n<\/td>\n

        		\n

        9.12<\/span><\/p>\n<\/td>\n

        		\n

        9.01<\/span><\/p>\n<\/td>\n

        		\n

        8.94<\/span><\/p>\n<\/td>\n

        		\n

        8.88<\/span><\/p>\n<\/td>\n

        		\n

        8.84<\/span><\/p>\n<\/td>\n

        		\n

        8.81<\/span><\/p>\n<\/td>\n

        		\n

        8.78<\/span><\/p>\n<\/td>\n

        		\n

        8.76<\/span><\/p>\n<\/td>\n

        		\n

        8.74<\/span><\/p>\n<\/td>\n

        		\n

        8.71<\/span><\/p>\n<\/td>\n

        		\n

        8.69<\/span><\/p>\n<\/td>\n

        		\n

        8.66<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        4<\/span><\/p>\n<\/td>\n

        		\n

        7.71<\/span><\/p>\n<\/td>\n

        		\n

        6.94<\/span><\/p>\n<\/td>\n

        		\n

        6.59<\/span><\/p>\n<\/td>\n

        		\n

        6.39<\/span><\/p>\n<\/td>\n

        		\n

        6.26<\/span><\/p>\n<\/td>\n

        		\n

        6.16<\/span><\/p>\n<\/td>\n

        		\n

        6.09<\/span><\/p>\n<\/td>\n

        		\n

        6.04<\/span><\/p>\n<\/td>\n

        		\n

        6.00<\/span><\/p>\n<\/td>\n

        		\n

        5.96<\/span><\/p>\n<\/td>\n

        		\n

        5.93<\/span><\/p>\n<\/td>\n

        		\n

        5.91<\/span><\/p>\n<\/td>\n

        		\n

        5.87<\/span><\/p>\n<\/td>\n

        		\n

        5.84<\/span><\/p>\n<\/td>\n

        		\n

        5.80<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        5<\/span><\/p>\n<\/td>\n

        		\n

        6.61<\/span><\/p>\n<\/td>\n

        		\n

        5.79<\/span><\/p>\n<\/td>\n

        		\n

        5.41<\/span><\/p>\n<\/td>\n

        		\n

        5.19<\/span><\/p>\n<\/td>\n

        		\n

        5.05<\/span><\/p>\n<\/td>\n

        		\n

        4.95<\/span><\/p>\n<\/td>\n

        		\n

        4.88<\/span><\/p>\n<\/td>\n

        		\n

        4.82<\/span><\/p>\n<\/td>\n

        		\n

        4.78<\/span><\/p>\n<\/td>\n

        		\n

        4.74<\/span><\/p>\n<\/td>\n

        		\n

        4.70<\/span><\/p>\n<\/td>\n

        		\n

        4.68<\/span><\/p>\n<\/td>\n

        		\n

        4.64<\/span><\/p>\n<\/td>\n

        		\n

        4.60<\/span><\/p>\n<\/td>\n

        		\n

        4.56<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        6<\/span><\/p>\n<\/td>\n

        		\n

        5.99<\/span><\/p>\n<\/td>\n

        		\n

        5.14<\/span><\/p>\n<\/td>\n

        		\n

        4.76<\/span><\/p>\n<\/td>\n

        		\n

        4.53<\/span><\/p>\n<\/td>\n

        		\n

        4.39<\/span><\/p>\n<\/td>\n

        		\n

        4.28<\/span><\/p>\n<\/td>\n

        		\n

        4.21<\/span><\/p>\n<\/td>\n

        		\n

        4.15<\/span><\/p>\n<\/td>\n

        		\n

        4.10<\/span><\/p>\n<\/td>\n

        		\n

        4.06<\/span><\/p>\n<\/td>\n

        		\n

        4.03<\/span><\/p>\n<\/td>\n

        		\n

        4.00<\/span><\/p>\n<\/td>\n

        		\n

        3.96<\/span><\/p>\n<\/td>\n

        		\n

        3.92<\/span><\/p>\n<\/td>\n

        		\n

        3.87<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        7<\/span><\/p>\n<\/td>\n

        		\n

        5.59<\/span><\/p>\n<\/td>\n

        		\n

        4.74<\/span><\/p>\n<\/td>\n

        		\n

        4.35<\/span><\/p>\n<\/td>\n

        		\n

        4.12<\/span><\/p>\n<\/td>\n

        		\n

        3.97<\/span><\/p>\n<\/td>\n

        		\n

        3.87<\/span><\/p>\n<\/td>\n

        		\n

        3.79<\/span><\/p>\n<\/td>\n

        		\n

        3.73<\/span><\/p>\n<\/td>\n

        		\n

        3.68<\/span><\/p>\n<\/td>\n

        		\n

        3.63<\/span><\/p>\n<\/td>\n

        		\n

        3.60<\/span><\/p>\n<\/td>\n

        		\n

        3.57<\/span><\/p>\n<\/td>\n

        		\n

        3.52<\/span><\/p>\n<\/td>\n

        		\n

        3.49<\/span><\/p>\n<\/td>\n

        		\n

        3.44<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        8<\/span><\/p>\n<\/td>\n

        		\n

        5.32<\/span><\/p>\n<\/td>\n

        		\n

        4.46<\/span><\/p>\n<\/td>\n

        		\n

        4.07<\/span><\/p>\n<\/td>\n

        		\n

        3.84<\/span><\/p>\n<\/td>\n

        		\n

        3.69<\/span><\/p>\n<\/td>\n

        		\n

        3.58<\/span><\/p>\n<\/td>\n

        		\n

        3.50<\/span><\/p>\n<\/td>\n

        		\n

        3.44<\/span><\/p>\n<\/td>\n

        		\n

        3.39<\/span><\/p>\n<\/td>\n

        		\n

        3.34<\/span><\/p>\n<\/td>\n

        		\n

        3.31<\/span><\/p>\n<\/td>\n

        		\n

        3.28<\/span><\/p>\n<\/td>\n

        		\n

        3.23<\/span><\/p>\n<\/td>\n

        		\n

        3.20<\/span><\/p>\n<\/td>\n

        		\n

        3.15<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        9<\/span><\/p>\n<\/td>\n

        		\n

        5.12<\/span><\/p>\n<\/td>\n

        		\n

        4.26<\/span><\/p>\n<\/td>\n

        		\n

        3.86<\/span><\/p>\n<\/td>\n

        		\n

        3.63<\/span><\/p>\n<\/td>\n

        		\n

        3.48<\/span><\/p>\n<\/td>\n

        		\n

        3.37<\/span><\/p>\n<\/td>\n

        		\n

        3.29<\/span><\/p>\n<\/td>\n

        		\n

        3.23<\/span><\/p>\n<\/td>\n

        		\n

        3.18<\/span><\/p>\n<\/td>\n

        		\n

        3.13<\/span><\/p>\n<\/td>\n

        		\n

        3.10<\/span><\/p>\n<\/td>\n

        		\n

        3.07<\/span><\/p>\n<\/td>\n

        		\n

        3.02<\/span><\/p>\n<\/td>\n

        		\n

        2.98<\/span><\/p>\n<\/td>\n

        		\n

        2.93<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        10<\/span><\/p>\n<\/td>\n

        		\n

        4.96<\/span><\/p>\n<\/td>\n

        		\n

        4.10<\/span><\/p>\n<\/td>\n

        		\n

        3.71<\/span><\/p>\n<\/td>\n

        		\n

        3.48<\/span><\/p>\n<\/td>\n

        		\n

        3.33<\/span><\/p>\n<\/td>\n

        		\n

        3.22<\/span><\/p>\n<\/td>\n

        		\n

        3.14<\/span><\/p>\n<\/td>\n

        		\n

        3.07<\/span><\/p>\n<\/td>\n

        		\n

        3.02<\/span><\/p>\n<\/td>\n

        		\n

        2.97<\/span><\/p>\n<\/td>\n

        		\n

        2.94<\/span><\/p>\n<\/td>\n

        		\n

        2.91<\/span><\/p>\n<\/td>\n

        		\n

        2.86<\/span><\/p>\n<\/td>\n

        		\n

        2.82<\/span><\/p>\n<\/td>\n

        		\n

        2.77<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        11<\/span><\/p>\n<\/td>\n

        		\n

        4.84<\/span><\/p>\n<\/td>\n

        		\n

        3.98<\/span><\/p>\n<\/td>\n

        		\n

        3.59<\/span><\/p>\n<\/td>\n

        		\n

        3.36<\/span><\/p>\n<\/td>\n

        		\n

        3.20<\/span><\/p>\n<\/td>\n

        		\n

        3.09<\/span><\/p>\n<\/td>\n

        		\n

        3.01<\/span><\/p>\n<\/td>\n

        		\n

        2.95<\/span><\/p>\n<\/td>\n

        		\n

        2.90<\/span><\/p>\n<\/td>\n

        		\n

        2.86<\/span><\/p>\n<\/td>\n

        		\n

        2.82<\/span><\/p>\n<\/td>\n

        		\n

        2.79<\/span><\/p>\n<\/td>\n

        		\n

        2.74<\/span><\/p>\n<\/td>\n

        		\n

        2.70<\/span><\/p>\n<\/td>\n

        		\n

        2.65<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        12<\/span><\/p>\n<\/td>\n

        		\n

        4.75<\/span><\/p>\n<\/td>\n

        		\n

        3.88<\/span><\/p>\n<\/td>\n

        		\n

        3.49<\/span><\/p>\n<\/td>\n

        		\n

        3.26<\/span><\/p>\n<\/td>\n

        		\n

        3.11<\/span><\/p>\n<\/td>\n

        		\n

        3.00<\/span><\/p>\n<\/td>\n

        		\n

        2.92<\/span><\/p>\n<\/td>\n

        		\n

        2.85<\/span><\/p>\n<\/td>\n

        		\n

        2.80<\/span><\/p>\n<\/td>\n

        		\n

        2.76<\/span><\/p>\n<\/td>\n

        		\n

        2.72<\/span><\/p>\n<\/td>\n

        		\n

        2.69<\/span><\/p>\n<\/td>\n

        		\n

        2.64<\/span><\/p>\n<\/td>\n

        		\n

        2.60<\/span><\/p>\n<\/td>\n

        		\n

        2.54<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        13<\/span><\/p>\n<\/td>\n

        		\n

        4.67<\/span><\/p>\n<\/td>\n

        		\n

        3.80<\/span><\/p>\n<\/td>\n

        		\n

        3.41<\/span><\/p>\n<\/td>\n

        		\n

        3.18<\/span><\/p>\n<\/td>\n

        		\n

        3.02<\/span><\/p>\n<\/td>\n

        		\n

        2.92<\/span><\/p>\n<\/td>\n

        		\n

        2.84<\/span><\/p>\n<\/td>\n

        		\n

        2.77<\/span><\/p>\n<\/td>\n

        		\n

        2.72<\/span><\/p>\n<\/td>\n

        		\n

        2.67<\/span><\/p>\n<\/td>\n

        		\n

        2.63<\/span><\/p>\n<\/td>\n

        		\n

        2.60<\/span><\/p>\n<\/td>\n

        		\n

        2.55<\/span><\/p>\n<\/td>\n

        		\n

        2.51<\/span><\/p>\n<\/td>\n

        		\n

        2.46<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        14<\/span><\/p>\n<\/td>\n

        		\n

        4.60<\/span><\/p>\n<\/td>\n

        		\n

        3.74<\/span><\/p>\n<\/td>\n

        		\n

        3.34<\/span><\/p>\n<\/td>\n

        		\n

        3.11<\/span><\/p>\n<\/td>\n

        		\n

        2.96<\/span><\/p>\n<\/td>\n

        		\n

        2.85<\/span><\/p>\n<\/td>\n

        		\n

        2.77<\/span><\/p>\n<\/td>\n

        		\n

        2.70<\/span><\/p>\n<\/td>\n

        		\n

        2.65<\/span><\/p>\n<\/td>\n

        		\n

        2.60<\/span><\/p>\n<\/td>\n

        		\n

        2.56<\/span><\/p>\n<\/td>\n

        		\n

        2.53<\/span><\/p>\n<\/td>\n

        		\n

        2.48<\/span><\/p>\n<\/td>\n

        		\n

        2.44<\/span><\/p>\n<\/td>\n

        		\n

        2.39<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        15<\/span><\/p>\n<\/td>\n

        		\n

        4.54<\/span><\/p>\n<\/td>\n

        		\n

        3.68<\/span><\/p>\n<\/td>\n

        		\n

        3.29<\/span><\/p>\n<\/td>\n

        		\n

        3.06<\/span><\/p>\n<\/td>\n

        		\n

        2.90<\/span><\/p>\n<\/td>\n

        		\n

        2.79<\/span><\/p>\n<\/td>\n

        		\n

        2.70<\/span><\/p>\n<\/td>\n

        		\n

        2.64<\/span><\/p>\n<\/td>\n

        		\n

        2.59<\/span><\/p>\n<\/td>\n

        		\n

        2.55<\/span><\/p>\n<\/td>\n

        		\n

        2.51<\/span><\/p>\n<\/td>\n

        		\n

        2.48<\/span><\/p>\n<\/td>\n

        		\n

        2.43<\/span><\/p>\n<\/td>\n

        		\n

        2.39<\/span><\/p>\n<\/td>\n

        		\n

        2.33<\/span><\/p>\n<\/td>\n<\/tr>\n

        \n

        16<\/span><\/p>\n<\/td>\n

        		\n

        4.49<\/span><\/p>\n<\/td>\n

        		\n

        3.63<\/span><\/p>\n<\/td>\n

        		\n

        3.24<\/span><\/p>\n<\/td>\n

        		\n

        3.01<\/span><\/p>\n<\/td>\n

        		\n

        2.85<\/span><\/p>\n<\/td>\n

        		\n

        2.74<\/span><\/p>\n<\/td>\n

        		\n

        2.66<\/span><\/p>\n<\/td>\n

        		\n

        2.59<\/span><\/p>\n<\/td>\n

        		\n

        2.54<\/span><\/p>\n<\/td>\n

        		\n

        2.49<\/span><\/p>\n<\/td>\n

        		\n

        2.45<\/span><\/p>\n<\/td>\n

        		\n

        2.42<\/span><\/p>\n<\/td>\n

        		\n

        2.37<\/span><\/p>\n<\/td>\n

        		\n

        2.33<\/span><\/p>\n<\/td>\n

        		\n

        2.28<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n

        Step 3:<\/span> Calculate the Test Statistic and Effect Size<\/h5>\n

        The process of calculating the test statistic for regression first involves computing the parameter estimates for the line of best fit. To do this, we first calculate the means, standard deviations, and sum of products for our X<\/span> and Y<\/span> variables, as shown below.<\/p>\n\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
        \n

        X<\/span><\/p>\n<\/td>\n

        		\n

        (X<\/span> \u2212 M<\/span>X<\/span>)<\/p>\n<\/td>\n

        		\n

        (X<\/span> \u2212 M<\/span>X<\/span>)2<\/span><\/p>\n<\/td>\n

        		\n

        Y<\/span><\/p>\n<\/td>\n

        		\n

        (Y<\/span> \u2212 M<\/span>Y<\/span>)<\/p>\n<\/td>\n

        		\n

        (Y<\/span> \u2212 M<\/span>Y<\/span>)2<\/span><\/p>\n<\/td>\n

        		\n

        (X<\/span> \u2212 M<\/span>X<\/span>)(Y<\/span> \u2212 M<\/span>Y<\/span>)<\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        17.65<\/p>\n<\/td>\n

        		\n

        \u22122.13<\/p>\n<\/td>\n

        		\n

        4.53<\/p>\n<\/td>\n

        		\n

        10.36<\/p>\n<\/td>\n

        		\n

        \u22127.10<\/p>\n<\/td>\n

        		\n

        50.37<\/p>\n<\/td>\n

        		\n

        15.10<\/p>\n<\/td>\n<\/tr>\n

        \n

        16.99<\/p>\n<\/td>\n

        		\n

        \u22122.79<\/p>\n<\/td>\n

        		\n

        7.80<\/p>\n<\/td>\n

        		\n

        16.38<\/p>\n<\/td>\n

        		\n

        \u22121.08<\/p>\n<\/td>\n

        		\n

        1.16<\/p>\n<\/td>\n

        		\n

        3.01<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.30<\/p>\n<\/td>\n

        		\n

        \u22121.48<\/p>\n<\/td>\n

        		\n

        2.18<\/p>\n<\/td>\n

        		\n

        15.23<\/p>\n<\/td>\n

        		\n

        \u22122.23<\/p>\n<\/td>\n

        		\n

        4.97<\/p>\n<\/td>\n

        		\n

        3.29<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.28<\/p>\n<\/td>\n

        		\n

        \u22121.50<\/p>\n<\/td>\n

        		\n

        2.25<\/p>\n<\/td>\n

        		\n

        14.26<\/p>\n<\/td>\n

        		\n

        \u22123.19<\/p>\n<\/td>\n

        		\n

        10.18<\/p>\n<\/td>\n

        		\n

        4.79<\/p>\n<\/td>\n<\/tr>\n

        \n

        21.89<\/p>\n<\/td>\n

        		\n

        2.11<\/p>\n<\/td>\n

        		\n

        4.47<\/p>\n<\/td>\n

        		\n

        17.71<\/p>\n<\/td>\n

        		\n

        0.26<\/p>\n<\/td>\n

        		\n

        0.07<\/p>\n<\/td>\n

        		\n

        0.55<\/p>\n<\/td>\n<\/tr>\n

        \n

        22.61<\/p>\n<\/td>\n

        		\n

        2.83<\/p>\n<\/td>\n

        		\n

        8.01<\/p>\n<\/td>\n

        		\n

        16.47<\/p>\n<\/td>\n

        		\n

        \u22120.98<\/p>\n<\/td>\n

        		\n

        0.97<\/p>\n<\/td>\n

        		\n

        \u22122.79<\/p>\n<\/td>\n<\/tr>\n

        \n

        17.42<\/p>\n<\/td>\n

        		\n

        \u22122.36<\/p>\n<\/td>\n

        		\n

        5.57<\/p>\n<\/td>\n

        		\n

        16.89<\/p>\n<\/td>\n

        		\n

        \u22120.56<\/p>\n<\/td>\n

        		\n

        0.32<\/p>\n<\/td>\n

        		\n

        1.33<\/p>\n<\/td>\n<\/tr>\n

        \n

        20.35<\/p>\n<\/td>\n

        		\n

        0.57<\/p>\n<\/td>\n

        		\n

        0.32<\/p>\n<\/td>\n

        		\n

        18.74<\/p>\n<\/td>\n

        		\n

        1.29<\/p>\n<\/td>\n

        		\n

        1.66<\/p>\n<\/td>\n

        		\n

        0.73<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.89<\/p>\n<\/td>\n

        		\n

        \u22120.89<\/p>\n<\/td>\n

        		\n

        0.79<\/p>\n<\/td>\n

        		\n

        21.96<\/p>\n<\/td>\n

        		\n

        4.50<\/p>\n<\/td>\n

        		\n

        20.26<\/p>\n<\/td>\n

        		\n

        \u22124.00<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.63<\/p>\n<\/td>\n

        		\n

        \u22121.15<\/p>\n<\/td>\n

        		\n

        1.32<\/p>\n<\/td>\n

        		\n

        17.57<\/p>\n<\/td>\n

        		\n

        0.11<\/p>\n<\/td>\n

        		\n

        0.01<\/p>\n<\/td>\n

        		\n

        \u22120.13<\/p>\n<\/td>\n<\/tr>\n

        \n

        19.67<\/p>\n<\/td>\n

        		\n

        \u22120.11<\/p>\n<\/td>\n

        		\n

        0.01<\/p>\n<\/td>\n

        		\n

        18.12<\/p>\n<\/td>\n

        		\n

        0.66<\/p>\n<\/td>\n

        		\n

        0.44<\/p>\n<\/td>\n

        		\n

        \u22120.08<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.39<\/p>\n<\/td>\n

        		\n

        \u22121.39<\/p>\n<\/td>\n

        		\n

        1.94<\/p>\n<\/td>\n

        		\n

        12.08<\/p>\n<\/td>\n

        		\n

        \u22125.37<\/p>\n<\/td>\n

        		\n

        28.87<\/p>\n<\/td>\n

        		\n

        7.48<\/p>\n<\/td>\n<\/tr>\n

        \n

        22.48<\/p>\n<\/td>\n

        		\n

        2.71<\/p>\n<\/td>\n

        		\n

        7.32<\/p>\n<\/td>\n

        		\n

        17.11<\/p>\n<\/td>\n

        		\n

        \u22120.34<\/p>\n<\/td>\n

        		\n

        0.12<\/p>\n<\/td>\n

        		\n

        \u22120.93<\/p>\n<\/td>\n<\/tr>\n

        \n

        23.25<\/p>\n<\/td>\n

        		\n

        3.47<\/p>\n<\/td>\n

        		\n

        12.07<\/p>\n<\/td>\n

        		\n

        21.66<\/p>\n<\/td>\n

        		\n

        4.21<\/p>\n<\/td>\n

        		\n

        17.73<\/p>\n<\/td>\n

        		\n

        14.63<\/p>\n<\/td>\n<\/tr>\n

        \n

        19.91<\/p>\n<\/td>\n

        		\n

        0.13<\/p>\n<\/td>\n

        		\n

        0.02<\/p>\n<\/td>\n

        		\n

        17.86<\/p>\n<\/td>\n

        		\n

        0.40<\/p>\n<\/td>\n

        		\n

        0.16<\/p>\n<\/td>\n

        		\n

        0.05<\/p>\n<\/td>\n<\/tr>\n

        \n

        18.21<\/p>\n<\/td>\n

        		\n

        \u22121.57<\/p>\n<\/td>\n

        		\n

        2.45<\/p>\n<\/td>\n

        		\n

        18.49<\/p>\n<\/td>\n

        		\n

        1.03<\/p>\n<\/td>\n

        		\n

        1.07<\/p>\n<\/td>\n

        		\n

        \u22121.62<\/p>\n<\/td>\n<\/tr>\n

        \n

        23.65<\/p>\n<\/td>\n

        		\n

        3.87<\/p>\n<\/td>\n

        		\n

        14.99<\/p>\n<\/td>\n

        		\n

        22.13<\/p>\n<\/td>\n

        		\n

        4.67<\/p>\n<\/td>\n

        		\n

        21.82<\/p>\n<\/td>\n

        		\n

        18.08<\/p>\n<\/td>\n<\/tr>\n

        \n

        19.45<\/p>\n<\/td>\n

        		\n

        \u22120.33<\/p>\n<\/td>\n

        		\n

        0.11<\/p>\n<\/td>\n

        		\n

        21.17<\/p>\n<\/td>\n

        		\n

        3.72<\/p>\n<\/td>\n

        		\n

        13.82<\/p>\n<\/td>\n

        		\n

        \u22121.22<\/p>\n<\/td>\n<\/tr>\n

        \n

        356.02<\/p>\n<\/td>\n

        		\n

        0.00<\/p>\n<\/td>\n

        		\n

        76.14<\/p>\n<\/td>\n

        		\n

        314.18<\/p>\n<\/td>\n

        		\n

        0.00<\/p>\n<\/td>\n

        		\n

        173.99<\/p>\n<\/td>\n

        		\n

        58.29<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        From the raw data in our X<\/span> and Y<\/span> columns, we find that the means are \"\" = 19.78 and \"\" = 17.45. The deviation scores for each variable sum to zero, so all is well there. The sums of squares for X<\/span> and Y<\/span> ultimately lead us to standard deviations of s<\/span>X<\/span> = 2.12 and s<\/span>Y<\/span> = 3.20. Finally, our sum of products is 58.29, which gives us a covariance of covXY<\/span> = 3.43, so we know our relationship will be positive. This is all the information we need for our equations for the line of best fit.<\/p>\n

        First, we must calculate the slope of the line:<\/p>\n

        \"\"<\/p>\n

        This means that as X<\/span> changes by 1 unit, Y<\/span> will change by 0.77. In terms of our problem, as health increases by 1, happiness goes up by 0.77, which is a positive relationship. Next, we use the slope, along with the means of each variable, to compute the intercept:<\/p>\n

        \"\"<\/p>\n

        For this particular problem (and most regressions), the intercept is not an important or interpretable value, so we will not read into it further. Now that we have all of our parameters estimated, we can give the full equation for our line of best fit:<\/p>\n

        Y<\/span>\u0302<\/span> = 2.42 + 0.77X<\/span><\/p>\n

        We can plot this relationship in a scatter plot and overlay our line onto it, as shown in Figure 13.3<\/span><\/a>.<\/p>\n

        \n
        \n

        <\/a>Figure 13.3.<\/span> Health and happiness data and line of best fit. (\u201c<\/span>Scatter Plot Health and Happiness<\/span><\/span><\/a>\u201d by Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n<\/div>\n<\/div>\n

        \n
        \"\"<\/div>\n<\/div>\n

        We can use the line equation to find predicted values for each observation and use them to calculate our sums of squares model and error, but this is tedious to do by hand, so we will let the computer software do the heavy lifting in that column of our ANOVA table:<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
        \n

        Error<\/p>\n<\/td>\n

        		\n

        129.37<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        Now that we have these, we can fill in the rest of the ANOVA table. We already found our degrees of freedom in Step 2:<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n
        \n

        Error<\/p>\n<\/td>\n

        		\n

        129.37<\/p>\n<\/td>\n

        		\n

        16<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        Our total line is always the sum of the other two lines, giving us:<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n
        \n

        Error<\/p>\n<\/td>\n

        		\n

        129.37<\/p>\n<\/td>\n

        		\n

        16<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		\n

        173.99<\/p>\n<\/td>\n

        		\n

        17<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        Our mean squares column is only calculated for the model and error lines and is always our SS<\/span> divided by our d<\/span>f<\/span>, which is:<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		<\/td>\n<\/tr>\n
        \n

        Error<\/p>\n<\/td>\n

        		\n

        129.37<\/p>\n<\/td>\n

        		\n

        16<\/p>\n<\/td>\n

        		\n

        8.09<\/p>\n<\/td>\n

        		<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		\n

        173.99<\/p>\n<\/td>\n

        		\n

        17<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        Finally, our F\u00a0<\/span>statistic is the ratio of the mean squares:<\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
        \n

        Source<\/p>\n<\/td>\n

        		\n

        SS<\/span><\/p>\n<\/td>\n

        		\n

        df<\/span><\/p>\n<\/td>\n

        		\n

        MS<\/span><\/p>\n<\/td>\n

        		\n

        F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

        \n

        Model<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		\n

        1<\/p>\n<\/td>\n

        		\n

        44.62<\/p>\n<\/td>\n

        		\n

        5.52<\/p>\n<\/td>\n<\/tr>\n

        \n

        Error<\/p>\n<\/td>\n

        		\n

        129.37<\/p>\n<\/td>\n

        		\n

        16<\/p>\n<\/td>\n

        		\n

        8.09<\/p>\n<\/td>\n

        		<\/td>\n<\/tr>\n
        \n

        Total<\/p>\n<\/td>\n

        		\n

        173.99<\/p>\n<\/td>\n

        		\n

        17<\/p>\n<\/td>\n

        		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

        This gives us an obtained F\u00a0<\/span>statistic of 5.52, which we will use to test our hypothesis.<\/p>\n

        Effect Size in Regression<\/h6>\n

        We know that, because our statistic is significant, we should calculate an effect size. In regression, our effect size is variance explained, just like it was in ANOVA. Instead of using h<\/span>2<\/span> to represent this, we instead use R<\/span>2<\/span>, as we saw in correlation\u2014yet more evidence that all of these are the same analysis. (Note that in regression analysis, R<\/span>2<\/span> is typically capitalized, although for simple linear regression it represents the same value as r<\/span>2<\/span> we used in correlation.) Variance explained is still the ratio of SS<\/span>M<\/span> to SS<\/span>T<\/span>:<\/p>\n

        \"\"<\/p>\n

        We are explaining 26% of the variance in happiness based on health, which is a large effect size. (R<\/span>2<\/span>\u00a0uses the same effect size cutoffs as h<\/span>2<\/span>.)<\/p>\n

        Step 4:<\/span> Make the Decision<\/h5>\n

        We now have everything we need to make our final decision. Our obtained test statistic was F <\/span>= 5.52 and our critical value was F<\/span>* = 4.49. Since our obtained test statistic is greater than our critical value, we can reject the null hypothesis.<\/p>\n

        Reject H<\/span>0<\/span>. Based on our sample of 18 people, we can predict levels of happiness based on how healthy someone is, and the effect size was large, F<\/span>(1, 16) = 5.52, p<\/span> < .05, R<\/span>2<\/span> < .26.<\/p>\n

        Figure 13.4<\/span><\/a> shows the output from JASP for this example.<\/p>\n

        \n
        \n

        <\/a>Figure 13.4.<\/span> Output from JASP for the linear regression described in the Happiness and Well-Being example. The output provides the slope (b<\/span>) of the line in the Coefficients table under Health Unstandardized (0.765), the y<\/span>-intercept (a<\/span>) of the line in the Coefficients table under (Intercept) Unstandardized (2.317, note slightly different from hand calculations due to rounding). The Model Summary \u2013 Happiness table provides R<\/span>2<\/span> (.256). The ANOVA table provides the result of the F<\/span> test statistic. Based on our sample of 18 people, we can predict levels of happiness based on how healthy someone is, F<\/span>(1, 16) = 5.5, p<\/span> = .032. (\u201c<\/span>JASP linear regression<\/span><\/span><\/a>\u201d by Rupa G. Gordon\/Judy Schmitt is licensed under <\/span>CC BY-NC-SA 4.0<\/span><\/span><\/a>.)<\/span><\/p>\n<\/div>\n<\/div>\n

        \n
        \"\"<\/div>\n<\/div>\n

        Accuracy in Prediction<\/h3>\n

        We found a large, statistically significant relationship between our variables, which is what we hoped for. However, if we want to use our estimated line of best fit for future prediction, we will also want to know how precise or accurate our predicted values are. What we want to know is the average distance from our predictions to our actual observed values, or the average size of the residual (Y<\/span> \u2212 Y<\/span>\u0302<\/span>). The average size of the residual is known by a specific name: the <\/a><\/a>standard error of the estimate<\/span> (\"\"), which is given by the formula<\/p>\n

        \"\"<\/p>\n

        This formula is almost identical to our standard deviation formula, and it follows the same logic. We square our residuals, add them up, then divide by the degrees of freedom. Although this sounds like a long process, we already have the sum of the squared residuals in our ANOVA table! In fact, the value under the square root sign is just the SS<\/span>E<\/span> divided by the d<\/span>f<\/span>E<\/span>, which is called the <\/a><\/a>mean squared error<\/span>, or MS<\/span>E<\/span>:<\/p>\n

        \"\"<\/p>\n

        For our example:<\/p>\n

        \"\"<\/p>\n

        So on average, our predictions are just under 3 points away from our actual values. There are no specific cutoffs or guidelines for how big our standard error of the estimate can or should be; it is highly dependent on both our sample size and the scale of our original Y<\/span> variable, so expert judgment should be used. In this case, the estimate is not that far off and can be considered reasonably precise.<\/p>\n

        Multiple Regression and Other Extensions<\/h3>\n

        Simple linear regression as presented here is only a stepping stone toward an entire field of research and application. Regression is an incredibly flexible and powerful tool, and the extensions and variations on it are far beyond the scope of this chapter. (Indeed, even entire books struggle to accommodate all possible applications of the simple principles laid out here.) The next step in regression is to study <\/a><\/a>multiple regression<\/span>, which uses multiple X<\/span> variables as predictors for a single Y<\/span>\u00a0variable at the same time. The math of multiple regression is very complex but the logic is the same: we are trying to use variables that are statistically significantly related to our outcome to explain the variance we observe in that outcome. Other forms of regression include <\/a><\/a>curvilinear models<\/span> that can explain curves in the data rather than the straight lines used here, as well as <\/a><\/a>moderation models<\/span> that change the relationship between two variables based on levels of a third. The possibilities are truly endless and offer a lifetime of discovery.<\/p>\n

        Exercises<\/h3>\n
          \n
        1. How are ANOVA and linear regression similar? How are they different?<\/li>\n
        2. What is a residual?<\/li>\n
        3. How are correlation and regression similar? How are they different?<\/li>\n
        4. What are the two parameters of the line of best fit, and what do they represent?<\/li>\n
        5. What is our criterion for finding the line of best fit?<\/li>\n
        6. Fill out the rest of the ANOVA tables below for simple linear regressions:\n
            \n
          1. \n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
            \n

            Source<\/p>\n<\/td>\n

            		\n

            SS<\/span><\/p>\n<\/td>\n

            		\n

            df<\/span><\/p>\n<\/td>\n

            		\n

            MS<\/span><\/p>\n<\/td>\n

            		\n

            F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

            \n

            Model<\/p>\n<\/td>\n

            		\n

            34.21<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
            \n

            Error<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
            \n

            Total<\/p>\n<\/td>\n

            		\n

            66.12<\/p>\n<\/td>\n

            		\n

            54<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n
          2. \n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
            \n

            Source<\/p>\n<\/td>\n

            		\n

            SS<\/span><\/p>\n<\/td>\n

            		\n

            df<\/span><\/p>\n<\/td>\n

            		\n

            MS<\/span><\/p>\n<\/td>\n

            		\n

            F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

            \n

            Model<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n\n

            6.03<\/p>\n<\/td>\n

            		<\/td>\n<\/tr>\n
            \n

            Error<\/p>\n<\/td>\n

            		<\/td>\n\n

            16<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/tr>\n
            \n

            Total<\/p>\n<\/td>\n

            		\n

            19.98<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<\/ol>\n<\/li>\n
          3. In Chapter 12<\/span><\/a>, we found a statistically significant correlation between overall performance in class and how much time someone studied. Use the summary statistics calculated in that problem (\"\" = 1.61, s<\/span>X<\/span> = 1.12, \"\" = 2.95, s<\/span>Y<\/span> = 0.99, r <\/span>= .65) to compute a line of best fit predicting success from study times.<\/li>\n
          4. Using the line of best fit equation created in Problem 7, predict the scores for how successful people will be based on how much they study:\n
              \n
            1. X<\/span> = 1.20<\/li>\n
            2. X<\/span> = 3.33<\/li>\n
            3. X<\/span> = 0.71<\/li>\n
            4. X<\/span> = 4.00<\/li>\n<\/ol>\n<\/li>\n
            5. You have become suspicious that the draft rankings of your fantasy football league have no predictive value for how teams place at the end of the season. You go back to historical league data and find rankings of teams after the draft and at the end of the season (shown in the following table) to test for a statistically significant predictive relationship. Assume SS<\/span>M<\/span> = 2.65 and SS<\/span>E<\/span>\u00a0=\u00a0337.35.
              \n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
              \n

              Draft Projection<\/p>\n<\/td>\n

              		\n

              Final Rankings<\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

              \n

              1<\/p>\n<\/td>\n

              		\n

              14<\/p>\n<\/td>\n<\/tr>\n

              \n

              2<\/p>\n<\/td>\n

              		\n

              6<\/p>\n<\/td>\n<\/tr>\n

              \n

              3<\/p>\n<\/td>\n

              		\n

              8<\/p>\n<\/td>\n<\/tr>\n

              \n

              4<\/p>\n<\/td>\n

              		\n

              13<\/p>\n<\/td>\n<\/tr>\n

              \n

              5<\/p>\n<\/td>\n

              		\n

              2<\/p>\n<\/td>\n<\/tr>\n

              \n

              6<\/p>\n<\/td>\n

              		\n

              15<\/p>\n<\/td>\n<\/tr>\n

              \n

              7<\/p>\n<\/td>\n

              		\n

              4<\/p>\n<\/td>\n<\/tr>\n

              \n

              8<\/p>\n<\/td>\n

              		\n

              10<\/p>\n<\/td>\n<\/tr>\n

              \n

              9<\/p>\n<\/td>\n

              		\n

              11<\/p>\n<\/td>\n<\/tr>\n

              \n

              10<\/p>\n<\/td>\n

              		\n

              16<\/p>\n<\/td>\n<\/tr>\n

              \n

              11<\/p>\n<\/td>\n

              		\n

              9<\/p>\n<\/td>\n<\/tr>\n

              \n

              12<\/p>\n<\/td>\n

              		\n

              7<\/p>\n<\/td>\n<\/tr>\n

              \n

              13<\/p>\n<\/td>\n

              		\n

              14<\/p>\n<\/td>\n<\/tr>\n

              \n

              14<\/p>\n<\/td>\n

              		\n

              12<\/p>\n<\/td>\n<\/tr>\n

              \n

              15<\/p>\n<\/td>\n

              		\n

              1<\/p>\n<\/td>\n<\/tr>\n

              \n

              16<\/p>\n<\/td>\n

              		\n

              5<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n

            6. You have summary data for two variables: how extroverted someone is (X<\/span>) and how often someone volunteers (Y<\/span>). Using these values, calculate the line of best fit predicting volunteering from extroversion, then test for a statistically significant relationship using the hypothesis-testing procedure: \"\" = 12.58, s<\/span>X<\/span> = 4.65, \"\" = 7.44, s<\/span>Y<\/span> = 2.12, r <\/span>= .34, N <\/span>= 67, SS<\/span>M<\/span> = 19.79, SS<\/span>E<\/span>\u00a0=\u00a0215.77.<\/li>\n<\/ol>\n
              \n
              \n

              Answers to Odd-Numbered Exercises<\/h3>\n<\/header>\n
              \n

              1)
              \nANOVA and simple linear regression both take the total observed variance and partition it into pieces that we can explain and cannot explain and use the ratio of those pieces to test for significant relationships. They are different in that ANOVA uses a categorical variable as a predictor, whereas linear regression uses a continuous variable.<\/p>\n

              3)
              \nCorrelation and regression both involve taking two continuous variables and finding a linear relationship between them. Correlations find a standardized value describing the direction and magnitude of the relationship, whereas regression finds the line of best fit and uses it to partition and explain variance.<\/span><\/p>\n

              5)
              \nLeast squares error solution; the line that minimizes the total amount of residual error in the dataset<\/span><\/p>\n

              7)
              \nb<\/span> = <\/span>r<\/span>*(<\/span>s<\/span>Y<\/span>\/<\/span>s<\/span>X<\/span>) = .65*(0.99\/1.12) = .57; <\/span>a<\/span> = <\/span>\"\" \u2212 <\/span>b<\/span>\"\" = 2.95 \u2212 (0.57)(1.61) = 3.87; <\/span>Y\u00a0<\/span>=\u00a03.87\u00a0+\u00a00.57<\/span>X<\/span><\/p>\n

              9)
              \nStep 1:<\/span> H<\/span>0<\/span>: <\/span>b<\/span> = 0 \u201cThere is no predictive relationship between draft rankings and final rankings in fantasy football,\u201d <\/span>H<\/span>A<\/span>: <\/span>b<\/span> \u2260 0 \u201cThere is a predictive relationship between draft rankings and final rankings in fantasy football.\u201d
              \n<\/span>Step 2:<\/span> Our model will have 1 (based on the number of predictors) and 14 (based on how many observations we have) degrees of freedom, giving us a critical value of <\/span>F<\/span>* = 4.60.
              \n<\/span>Step 3:<\/span> Using the sum of products table, we find <\/span>\"\" = 8.50, <\/span>\"\" = 8.50, <\/span>SS<\/span>X<\/span> = 339.86, <\/span>SP<\/span> = 29.99, giving us a line of best fit of: <\/span>b<\/span> = 29.99\/339.86 = 0.09; <\/span>a<\/span> = 8.50 \u2212 (0.09)(8.50) = 7.74; <\/span>Y\u00a0<\/span>=\u00a07.74\u00a0+\u00a00.09<\/span>X<\/span>. Our given <\/span>SS<\/span> values and our <\/span>d<\/span>f<\/span> from Step 2 allow us to fill in the ANOVA table:<\/span><\/p>\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
              \n

              Source<\/p>\n<\/td>\n

              		\n

              SS<\/span><\/p>\n<\/td>\n

              		\n

              df<\/span><\/p>\n<\/td>\n

              		\n

              MS<\/span><\/p>\n<\/td>\n

              		\n

              F<\/span><\/p>\n<\/td>\n<\/tr>\n<\/thead>\n

              \n

              Model<\/p>\n<\/td>\n

              		\n

              2.65<\/p>\n<\/td>\n

              		\n

              1<\/p>\n<\/td>\n

              		\n

              2.65<\/p>\n<\/td>\n

              		\n

              0.11<\/p>\n<\/td>\n<\/tr>\n

              \n

              Error<\/p>\n<\/td>\n

              		\n

              337.35<\/p>\n<\/td>\n

              		\n

              14<\/p>\n<\/td>\n

              		\n

              24.10<\/p>\n<\/td>\n

              		<\/td>\n<\/tr>\n
              \n

              Total<\/p>\n<\/td>\n

              		\n

              339.86<\/p>\n<\/td>\n

              		\n

              15<\/p>\n<\/td>\n

              		<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

              Step 4:<\/span> Our obtained value was smaller than our critical value, so we fail to reject the null hypothesis. There is no evidence to suggest that draft rankings have any predictive value for final fantasy football rankings, F<\/span>(1, 14) = 0.11, p<\/span> > .05<\/p>\n<\/div>\n<\/div>\n

               <\/p>\n

              definition<\/span>