2
$\begingroup$

I understand that in observational studies, if we want to use regression for analysis, and if some variables are supposed to be a confounding variable, we should control for them in the regression model. However, I am a bit confused about adding covariates in ANOVA kind of analysis. Here is the scenario:

Scenario: In a randomised experiment in which participants are randomly assigned into 3 different treatment groups, and then we measure their performance (e.g., response times) on some task as our only DV. However, participants' IQ and EQ are supposed to be covariates, so they were measured prior to the treatment.

Question1: In general, should we run ANOVA or ANCOVA?

Question2: If we indeed found no significant difference in participants' IQ, shall we treat IQ as covariate in our model?

Question2: If we unfortunately find that the 3 groups have different levels of EQ even though we used random assignment, should we run ANOVA or ANCOVA?

Edit 1: Assume that the observations are independent of each other by random sampling. Also assume that the distribution of DV is normal.

Improve this question
$\endgroup$
2
  • $\begingroup$ Please read, Miller and Chapman (2001): Misunderstanding analysis of covariance. It talks exactly about your question: bwgriffin.com/gsu/courses/edur8132/notes/… I think the general answer will be, no ANCOVA. Explained in the paper. $\endgroup$
    – Henrik
    Commented Feb 3, 2018 at 10:48
  • $\begingroup$ Hmm, maybe my initial no was the wrong answer as the Miller and Chapman mostly deal with non-random assignment (IIRC). However, read it and it will most likely answer your question. (+1) $\endgroup$
    – Henrik
    Commented Feb 3, 2018 at 10:59

2 Answers 2

Reset to default
1
$\begingroup$

Questions on ANOVA can be found here: ANOVA assumption normality/normal distribution of residuals

I am guessing this is homework, so I will offer some advice on regression analysis. The first step is to run a summary statistics on your data (univariate analyses). This will give you important means, std, variance information that you can use to qualify the ANOVA assumptions (there are three that are found in the above link). Then, I would run a correlation study to see what variables are correlated and how much. After that, you can start thinking about ANOVA or a multiple linear regression (this should also provide you with an Analysis of Variance table that gives you a p-value and an F-statistic. I use SAS and mine looks like this: SAS Linear Regression Example

This is how I write it up.

The number of observations in the study is 3987. There is a significant relationship between age and education (IVs) and wages (DV), {F (2, 3984) = 660.71, p<0.0001 and according to r^2, 24.91% of the variance in wages is attributed to age and education}. We reject the null. Age represents the strongest predictor to wages as indicated by t=28.72; CI: 0.24, 0.27. Education (t=25.21; CI: 0.83, 0.97), also showed a significant positive linear relationship to wages. The model is a good fit based on the significant positive relationship that education and age (IVs) have with wages (DV). The r-square value could be higher (>50%) to better explain the variance that each IV has on the DV, but this is still a good model fit with a high F-value.

Edit: I wanted to add that if the distributions are not normal, then ANOVA does not apply. You can then think about PCA or Factor Analysis.

Edit 2: I forgot to address your main concern, I apologize. You can run a 'between-groups' ANOVA if the two groups have no common participants because they meet the independence requirement 1. If you are running a longitudinal study on a randomly selected group, you could use a repeated measures ANOVA [2].

1 http://www.statisticssolutions.com/difference-between-random-selection-and-random-assignment/ [2] https://statistics.laerd.com/statistical-guides/repeated-measures-anova-statistical-guide.php

Hope this helps!

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ You do not seem to address the OP's main question about random assignment? Do you want to edit that in? $\endgroup$
    – mdewey
    Commented Jan 29, 2018 at 17:29
  • $\begingroup$ Thank you for your suggestion (but this is not homework... i just want to clarify my confusion). So you suggest ANOVA but not ANCOVA? Why? $\endgroup$
    – JetLag
    Commented Jan 30, 2018 at 0:56
1
$\begingroup$

I would think ANCOVA because:

1) You seem to have thought (a priori) that IQ and EQ could be important and the analysis should reflect that. In some fields of research, it is expected that all of the analyses are determined a priori and all of the measured variables are reported for ethical reasons (based on the description, I’m assuming this is psychology. See Psychological Science’s policies about reporting on all variables).

2) Even if there was no bivariate relationship between IQ/EQ and response times, it is technically possible for a significant relationship to appear when multiple covariates are included (for an example, Google “suppressor variables”).

3) Even if you randomize, it is possible that there will be some residual confounding between IQ/EQ and your intervention groups. In your field, it may be expected that you control for that. Even if IQ/EQ is not confounded with your intervention, you may be able to reduce noise/increase power by using an ANCOVA (see Miller & Chapman (2001). Misunderstanding Analysis of Covariance. Journal of Abnormal Psychology, 110, 40-48)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.