3
$\begingroup$

I hope you can steer me in the right direction. I'm helping a colleague analysing a dataset that consists of:

  • one dependent variable (value) measured 3 times (observation) by 3 people (evaluator) on the 2 sides (side - right/left) of an individual (~150 individuals)

  • several independent variables: one continuous (age), and 2 discrete (sex, ancestry).

  • the data also has NA has some evaluators skipped over a few individuals in one or more of the observation points.

    enter code here

His main questions concern the reproducibility of measurements and effect of independent variables:

  • for reproducibility I used intraclass correlation coefficients with relative success;
  • to look at the effect of independent variables, I would normally use some form of a linear model (ANOVA/ANCOVA).

However, I'm worried about the lack of independence between observations, basically the first assumption of a lot of these tests. I tried looking at mixed models (e.g. using 1 between subjects factor (sex) and 2 within subjects factors (evaluator & observation) ), but I couldn't find good examples of this application (particularly using R), and mixed models seem to focus on evaluating the impact of these within subject factors, when in fact I mostly just want to control for these and focus on the effect of the independent factors.

Questions: (1) Could I assume independence? (2) What would be an appropriate design model (@RobertLong 's correction)? (3 - slightly off topic) Any R pointers are welcome.

Update Following Robert Long's answer, I'm now working with a mixed model that looks like this lme(value ~ sex * side + age + ancestry + evaluator, data = L, random = ~1|individual, method = "ML", na.action = na.exclude) (using Andy Field's book)

However, now I have more questions: (1) How to follow up (especially considering that I want to understand the effect of sex while controlling for the effect of ´evaluator´. Would contrast be a good option? (2) Reading on this, I realised that lack of independence will also be an issue for the ICC I calculated. Could I calculate them from the mixed model? (Based on this: Computing repeatability of effects from an lmer model)

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ How many individuals do you have ? What is the "evaluator" variable (is that the 3 different people doing the measurements ?) Is it the same 3 people all the time, or are the more than 3 people (ie 3 different people for other individuals) ? $\endgroup$
    – Robert Long
    Jul 7, 2021 at 18:28
  • $\begingroup$ I edited the question to include the very pertinent info you requested. In summary: around 150 individuals; Yes; Yes. I appreciate any help. I'm used to the typical frequentist statistics but getting lost in a mixed model rabbit hole. Thank you. $\endgroup$
    – answer42
    Jul 7, 2021 at 19:07

1 Answer 1

Reset to default
5
$\begingroup$

(1) Could I assume independence?

No, because you have repeated measurements. Observations within one individual are more likely to be similar to other observations within the same individual than observations on other individuals. One way to handle this is to fit random intercepts for individuals in a mixed model. It also seems that evaluator is crossed with individual, but you don't have enough evaluators to justify fitting random intercepts for that, so you will need to fit fixed effects for evaluator.

(2) What would be an appropriate design?

Perhaps you mean model, not design ? The (study) design is the experiment itself which apparently has already been done. As for the model, a mixed model may be appropriate.

3 - (slightly off topic) Any R pointers are welcome.

I would start with a model such as:

value ~ age + sex + ancestry + side + evaluator + (1 | individual)

You could use the lmer function in the lme4 package for that.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Got it for question 1 and 2. Thank you. Re. R: (1) Should I ignore observation? Maybe +observation*evaluator or is this not appropriate? (2) is it okay to convert factors to numeric? To get a single p-value per variable (3) [trickier as you don't have my data] I get a surprising significant p-value for side (absent visually and when ignoring data structure) prob bc sex is unbalanced so could include sex*size (significant btw)? $\endgroup$
    – answer42
    Jul 7, 2021 at 21:06
  • 1
    $\begingroup$ observation is the lowest level - there are no repeated measures within in it right ? if so then you don't need to include it. observation*evaluator if you expect an interaction between them, though based on the description I can't see why what would be. Factors to numeric: that usually doesn't make sense. With a binary variable it doesn't matter but if it's categorical with more than 2 levels then it does not make sense. As for p-values please ask a different question about, but p-values for mixed models are not well defined to begin with so please try not to be too concerned with them. $\endgroup$
    – Robert Long
    Jul 7, 2021 at 21:48
  • $\begingroup$ No repeated measures within observation; Factors to numeric: I was following Andy Field's book which suggests this as a way to evaluate effect of categorical variable overall rather than getting a different result for each factor level $\endgroup$
    – answer42
    Jul 8, 2021 at 14:11
  • $\begingroup$ Please ask a seperate question about treating categorcal variables as numeric as that is a different question. $\endgroup$
    – Robert Long
    Jul 8, 2021 at 14:21
  • 1
    $\begingroup$ So, to "understand the effect of sex while controlling for the effect of evaluator" you just need to fit a model that includes both sex and evaluator plus any other competing exposures, such as age and ancestry, so the model I suggested would be appropriat for that. As for the ICC, you can calculate that directly from the mixed model. As for independence, by fitting random intercepts for individual and fixed effects for side and evaluator, we are contolling for non-independence. $\endgroup$
    – Robert Long
    Jul 8, 2021 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.