0
$\begingroup$

I have a question about ANCOVA.

We did an experiment with 2 groups and then calculated an ANCOVA to see if the DV differed between the groups.

We included one binary and one continous covariate (PO). Both did not sig. differ between groups.

It was criticized that PO is skewed - we have more participants with a low PO score than those with a high PO score. It was suggested that we resample and specifically recruit those with a high PO score.

I got the tip to argue that such a change in covariate (e.g., getting closer to a normal distribution) would probably not bring major changes in the result, only smaller standard errors. However, the person did not know the exact reasoning for this anymore.

Can someone here perhaps give me a hint how I could justify this and/or literature tips on this? My research has been unsuccessful so far....

Edit: The distribution was also criticized because the group with a high PO score was relatively small - not enough in the eyes of the reviewer. I didn't make that clear before.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ AN(C)OVA, and linear models more generally, make no assumptions about the distribution of IVs (and far fewer assumptions about the distribution of DVs than is commonly assumed). I cannot think of a statistical reason to be concerned about the skew in an IV, and I would say it is incumbent on your reviewer to explain why they think their proposal makes sense. In the meantime, of course resampling will mean that your sample is not representative any more. See Miller & Chapman (2001). $\endgroup$
    – Stephan Kolassa
    Commented Aug 24, 2021 at 15:46
  • $\begingroup$ The skew does not bother me. As long as you have "enough" (I'll leave that vague) members in the smaller group, you should be good. $\endgroup$
    – Dave
    Commented Aug 24, 2021 at 15:48
  • $\begingroup$ Thank you for the comments! I think Dave's point is the reason why the reviewer was critical - not enough people in the smaller group. I was tol there was a statistical reason for why resampling more people in this group would probably not affect the main effect... $\endgroup$
    – Joe55711
    Commented Aug 24, 2021 at 16:18
  • $\begingroup$ Where you talk about covariates you say one is 'bivariate'. Do you mean binary (taking two possible values)? $\endgroup$
    – Glen_b
    Commented Aug 24, 2021 at 16:27
  • $\begingroup$ yes, sorry, binary! hooops $\endgroup$
    – Joe55711
    Commented Aug 24, 2021 at 16:54

1 Answer 1

Reset to default
0
$\begingroup$

ANOVA/ANCOVA(sig. tests of regression coefficients) does not require that IVs are normally distributed. The only assumption of normality is for the residuals. In terms of the benefits of ensuring your IV is not skewed, the only justifiable reason I can think of for this would be if you had reason to believe that your sample was not representative of the target population. To make up an example... suppose you collected data on college studying behavior (in hours/week; PO) as it predicts GPA. When you collected your sample, however, you surveyed mostly first-year college students and undersampled more acclimated students. As a result (perhaps) you have a lot of people with very little studying hours/week, and only a handful with a lot of hours. However, this sampling issues is going to apply to ALL of your variables, not just your PO IV. If you resample for one of your variables, you should resample for all.

In the end what you want from your sample is for it to be representative of your target population. Without getting too into the weeds here, one way to do so, without adding bias at least, is to take a truly random sample of the population. You may end up with a weird sample, but on the average your samples will equal the target population, and you won't have added a systematic bias to them. However, many times we can't really do a truly random sample, so we end up doing the best we can. Stratified sampling might work for you. Returning to our college example, maybe you want to ensure that your sample contains 25% 1st year students, 25% 2nd year students, etc. (simplified example in which there are only 4 years possible).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your detailed answer! I agree, a representative sample would be great. Due to some difficulties during the data collection (e.g., COVID...), our sample is slightly more right-skewed than the population. I got a tip, however, that including more particpants with a high PO, thus at the other extreme of the PO scala, would propably not change the main effect of the ANCOVA, since the 'pattern of results' would be unchanged. But I can't find anythign substantial to support that claim - other than that covariates don't need to be normally distributed. $\endgroup$
    – Joe55711
    Commented Aug 25, 2021 at 13:08
  • $\begingroup$ If you are concerned that your sample is not representative of the intended population, then make sure to indicate this to readers. As for whether purposely sampling participants with high PO, and including that new sample in your data, would change results, that's an empirical question. I image you have a a small number of such people based on your other comments, but what does their data look like? Do they tend to have higher or lower DV values than those with lower PO? You won't be able to build a compelling case without adequate data, but you can at least see what your data suggest. $\endgroup$
    – JRP
    Commented Aug 25, 2021 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.