0
$\begingroup$

Is the best way to compare the intervention group (n=32) with the control group (n=32) to do a t-test? I've got pre-test and post-test results and did t-test of both the pre-scores (IG compared to CG) and then later the post-scores (IG compared to CG). Do I need to investigate normal distribution by a bell chart although I'm just investigating a small group that is selected out of convenience?

Should I have done an ANOVA instead?

$\endgroup$
2
  • 1
    $\begingroup$ Welcome to Cross Validated. Please provide more information about your data & hypothesis. For example: What is the outcome variable? What do you mean by "selected out of convenience"? Do you only have pre- and post-scores or you have other (potentially) relevant measurements? $\endgroup$
    – dipetkov
    Apr 28, 2022 at 5:19
  • $\begingroup$ Linear mixed models could be a possible solution. Here is an example of lmm with STATA, SAS, and R. I hope it helps. $\endgroup$
    – Roberto
    May 3, 2022 at 10:15

1 Answer 1

-1
$\begingroup$

If I am correct, you have a control study that has a control group where something was measured, then a treatment happened, and something was measured again. If so, than the example data set below can be a step-by-step guide to help you know if the t-test is the right test?? And what are a few of the different ways of knowing how to do this.

Since you did not provide any data, I made one up in R.

Let's pretend if I want to know if coffee consumption increases alertness of professors teaching a statistics class. There will be a measurement before coffee consumption, and their alertness is measured on a scale of 0-200, where 200 = really alert. Then, the same person drinks coffee, and their alertness is then rated again. I do all of this for 32 different professors (N=32 professors, N=64 total measurements (each one measured twice, don't forget)

Here's a pretend dataset

# Create an empty data frame with column names
example_df <- data.frame( "ID" = character(0), "alertness" = integer(0))
#Assign names to x 
variable_names <- c("control", "after_intervention")
# Assign names to y
w<-rnorm(32, mean=10, sd=2)
x<-rnorm(32, mean=8, sd=2)
#combine everything to create dataframe (df)
df <- data.frame( "ID" = variable_names, "alertness" = c(w,x))

Can I do a t-test? Does my data meet the normality assumptions?

  • We can do this by checking quantile-quantile plots with the qqPlot() command from the library(car)
  • We can run a shapiro test to see if we meet normality assumptions
  • We can make histograms of our data and make a decision based on the shape.
  • There are other options too, please ask if interested

quantile-quantile plots

library(dplyr)

control <- filter(df, ID=="control")
after_intervention <- filter(df, ID=="after_intervention")


library(car)


par(mfrow = c(1,2), cex = .8)

qqPlot(control$alertness, dist = "norm",
       mean = mean(control$alertness), sd = sd(control$alertness),
       xlab = "Theor. quantiles (norm)", ylab = "Empirical quantiles", main = "before")

qqPlot(after_intervention$alertness, dist = "norm",
       mean = mean(after_intervention$alertness), sd = sd(after_intervention$alertness),
       xlab = "Theor. quantiles (norm)", ylab = "Empirical quantiles", main = "after")

enter image description here

The qqPlots actually look pretty good, and it seems the normality assumption is met. But what if I want to do another test to check. Lets do a shapiro test


shapiro.test(control$alertness)
shapiro.test(after_intervention$alertness)

enter image description here

The p-value of both tests is not less than .05, which indicates that the data is normally distributed.

Ok, but now I want a third way of knowing, lets do some histograms

hist(control$alertness)
hist(after_intervention$alertness)

enter image description here

Also looks fine.

So, here you could do a paired t-test, paired because we measured the same subject (same professor). It would be unpaired if this was not done (aka we measured different professors)

To do an anova, you can directly put your data in a linear model


model <- lm(alertness~ID, data = df)
summary(model)

enter image description here

And at the bottom, you see the anova summary, with f-statistic, and p-value.

Finally, you can see that the linear model truly did to a proper anova, but just literally running an anova, and seeing that the results match, by running the following code


difference <-aov(df$alertness~df$ID)
summary(difference)

Hope that helps

$\endgroup$
9
  • 1
    $\begingroup$ why down vote my answer without an explanation? All information I provided are studied methods to check distribution $\endgroup$
    – Andy
    May 3, 2022 at 11:08
  • $\begingroup$ It's frustrating to get a downvote and no feedback, so here are a couple of suggestions (though it I didn't downvote your answer). To start with, the OP provides very little (insufficient?) information, so you basically come up with your own question. Next time consider whether there is enough substance to the question to warrant an answer. $\endgroup$
    – dipetkov
    May 3, 2022 at 11:27
  • $\begingroup$ You ignore the single piece of information provided: that the sample sizes are 32 for both groups. This matters because you then proceed to check normality and this is easier with a sample of size 200. The coffee/alertness example probably doesn't have anything to do with the OP's actual domain, so if you want to simulate data keep it abstract and call it x and y. Finally, I don't think tests for normality are a great idea but let's call this difference in opinion about statistical practice. $\endgroup$
    – dipetkov
    May 3, 2022 at 11:27
  • $\begingroup$ @dipetkov I agree that they have only 32 data points / treatment. Nonetheless, the same steps can be applied as I mentioned. The question is lacking info, I agree, which is why I made up a dataset to help visualize my response, but it's true the OP lacked to provide us with sufficient information to maybe even merit a response. Finally, the coffee/alertness just shows a setup with a baseline (control) and manipulation (intervention). I just made up names, but tried to respect the authors original question. I hope this clarifies any/all points. All the best. $\endgroup$
    – Andy
    May 3, 2022 at 11:33
  • $\begingroup$ Using n = 200 instead of n = 32 to do a hypothesis test is not the best illustration because the power of a test increases with sample size. $\endgroup$
    – dipetkov
    May 3, 2022 at 11:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.