Between-Subjects One-Way ANOVA

In our first hypothetical experiment, participants attended a lecture and were asked to come back the next day to take a test on the lecture material after having received either 0, 5, or 8 hours of sleep. Therefore, sleep is manipulated between participants. Here we are interested in the effect of hours slept on test scores, we can do this by conducting a one-way analysis of variance (ANOVA).

A one-way ANOVA is an extension of an independent two-sample t-test. Here, the data is organized into several groups of participants (e.g., 3 groups) based on a single grouping variable, also called a factor variable (e.g., sleep). You can think of a factor as an independent variable with more than 2 levels of manipulation. In our case, we have three different levels of manipulation to sleep.

Therefore, in a one-factor ANOVA, we are comparing means in a situation where there are more than two levels to a single factor. The aim of this tutorial is to describe the basic principle of the one-way ANOVA test and provide practical ANOVA test examples in R software.

NOTE: if you have only two groups, an F-test and a t-test are the same.

In an ANOVA we are testing two hypotheses: 1. Null hypothesis: there is no difference between the means of the different groups 2. Alternative hypothesis: there is a difference in means between at least two groups.

Now, we aim to reject the null hypothesis in favour of the alternative, by showing that there is a significant difference between at least 2 groups of participants.

The first step in doing so is to import our data set, then inspect our file.

Note that in R studio, once you load your data file, you will notice it pop up under the name you saved it as, which in this case is “mydata1” at the top right corner of your screen, which is your global environment. Here you will see all your stored work, values, and files. You can click on the data file under data to open it as a separate tab beside your script, to look through, or you can do this exact action by code using view(). I tend to use this function when my global environment gets super packed, and it is no longer feasible to scan through the multitude of data for a specific one I want to take a look at.

Now in this data file, ID distinguishes between individual participants.

We also have some basic demographic info, including “Age” and “Gender”, which we typically summarize and report when describing our sample size in our Methods section. These demographic info may differ depending on your test subjects, but these are the typical ones we report with human subjects.

In this data file, our numerical dependent variable is in percentage form out of 100% (“TestScore”).

Our factor is in the column “Sleep”. Let’s check out to see if the factor is set up correctly by looking at the structure of our data frame.

We can see that our dependent measure “TestScore” is set up correctly as numerical. If not, then we need to fix it by using the following code:

Note that by using “$” beside our data frame name, we are calling upon a column name, as you start typing you will notice that a list of all the column names comes up as a short-cut.

We can also see that ID and Sleep are not set-up as factors. Let’s fix that.

Summary Statistics

Next, let’s compute some summary statistics for each group across the different levels of Sleep.

Here I am looking for group means along with the standard error of the mean values, which I will later use for my error bars when making figures. Some other measures that you can use are listed under the info card for the get_summary_stats function and include: type = c(“mean_sd”, “mean_se”, “mean_ci”, “median_iqr”, “median_mad”, “quantile”, “mean”, “median”, “min”, “max”)

For example, you may be interested in using mean standard deviation values for your error bars, in which case you can use “mean_sd”. You may also use “full” as your type to get all the measures!

Pro Tip: You can explore what each function offers with examples by putting a “?” right before the empty function, as shown below. It will open up the information sheet for the function on the Help tab to your bottom right on R studio. Try it!

Making Publishable Tables

Now, while this is a nice table if you want to include these summary statistics in your final thesis paper. We can do the following steps to generate nice printable summary tables!

Reporting Demographic Info As I mentioned before, one of the first things we have to do when writing up our Methods is to report some basic information about our sample size. Let’s do that now!

Note that n represents the number of participants per group, meaning that in total, we have data from 258 participants. More importantly, we can see that the number of observations per condition is equal, therefore, this is a balanced design. If you have an unequal sample size, I recommend that you refer to the following link for more information on conducting an unbalanced ANOVA:

How to perform one-way anova with unequal sample sizes in R?

However, I would still recommend reading through this guide to become familiar with R-studio. You will be doing the same steps when doing an unbalanced ANOVA, with some adjustments to the code.

Reporting Participants

Here is an example of how you might report information about your sample size in your Methods section based on the above summary tables.

Example: We recruited 258 participants from the introductory psychology undergraduate participant pool at McMaster University in exchange for partial course credit. Participants were randomly assigned to either the no sleep condition (39 males and 10 non-binary; Mean age = 23.7 years), 5 hour sleep condition (n=86; 40 males and 10 non-binary; Mage = 24.0 years), or the 8 hour sleep condition (n=86; 33 males and 8 non-binary; Mage = 24.0 years)

Visualization: Generating Plots & Figures

With our summary statistics in hand. Let’s generate some nice summary plots to visualize our data!=

LINE PLOT

The first one we will create is a line plot. So here we want to specify which factor we want on our x-axis, as well as which summary measure of our dependent variable we want on our y-axis. In this case, we want the Sleep factor (with 3 levels) on the x-axis, and mean Test Scores on the y-axis

Tip: One thing to keep in mind is that you should design your plots such that if a reader is colour-blind or chooses to print your results in black-white, they can still easily distinguish between the different factors in your plot. Hence, we will also use the shape of our points to also specify “BLOCK_TYPE”

 

BOX PLOT

BAR PLOT

If you are interested in exploring your options when it comes to bar plots, I recommend the following resources:

• Step-by-Step Barplots for One Factor in R
• ggplot2 barplots : Quick Start Guide
• This post explains how to draw barplots with R and ggplot2, using the geom_bar() function.
• Elegant barplot using ggplot function in R

The following link will show you all the different colour palettes that you can use for designing plots.

• ggplot2 Reference and Examples (Part 2) – Colours.

Computing a One-Way ANOVA

Interpreting & Reporting

We can see that the p-value (Pr(>F)) is significantly less than the significance level of 0.05, as highlighted by the “asterisk” symbols in the summary table. Therefore we can reject the null hypothesis that there were no differences between groups. When reporting these data, we typically have to report the degrees of freedom, the F value, p-value, and a measure of effect size). Next we are going to calculate effect sizes, in specific, we will be reporting partial eta squared here. You should consult with your principal investigator on which measure of effect size they deem appropriate.

For a more comprehensive review of the effect size measure options please go to the following resource: Effect Sizes for ANOVAs

Pair-Wise Comparisons

Next, while our one-way ANOVA showed that there is a significant difference between groups, we don’t know which pairs of groups are different. Using multiple pairwise comparisons, we can probe where this difference lies between pairs of groups.

In R, there are multiple functions for conducting the same analyses. My personal favourite for this next step is the function TukeyHD(), which takes our fitted one-way ANOVA as an argument.

When interpreting this output:

• diff: the difference between the means of the two groups
• lwr and upr correspond to the lower and the upper-end point of the confidence interval at 95%.
• p adj: p-value after adjustment for the multiple comparisons

It can be seen from the output that there is a significant difference between all combinations of the groups! However, the biggest benefit in performance occurs when participants receive 8 hours of sleep compared to no sleep.

Together with our ANOVA summary table, we now have everything we need to report our results.

Reporting Results:

The test score was statistically significantly different at the different amounts of sleep [F(2, 255) = 117.4, p < 0.001, pη² = 0.48]. Post-hoc analyses with a Bonferroni adjustment revealed that all the pairwise differences, between sleep levels, were statistically significantly different (p < 0.001).