5
$\begingroup$

This is the experimental setup:

1 dependent variable (discrete, 4 levels) and

3 Independent variables:

  • Time, measured within subject, 5 discrete levels
  • Covariate, measured within subject, 5 discrete levels
  • Treatment, measured between subject, 5 discrete levels

Research question: What is the effect of Treatment on the dependent variable?

I plan to use a multi-level regression model where I first regress time and the covariate on the outcome variable(level1) and then use the resulting regression coefficients as dependent variables in level 2, as is done here: http://joophox.net/mlbook2/Chapter2.pdf or https://en.wikipedia.org/wiki/Multilevel_model

Level 1: Outcome = b0 + b1 * Time + b2 * Covariate +e

Level 2: b0 = g0 + g1 * Treatment + u and b1 = g0 + u

As far as I understand, the coefficient g1 is the one that would answer my research question. If it's significant then the treatment has an effect on the outcome.

Now I have a problem with the intercept b0. b0 is the mean of the outcome when the predictor is = 0. However, my time variable is discrete, in 1-5 steps. I have difficulties imagining how I can make this meaningful, so that it makes sense in the second level. If I center Time, then I get (-2,-1,0,1,2), so in essence just the mean outcome at the third time point? But what I need for the level 2 regression to be meaningful is (I think) the mean of the outcome variable across all time points.

Is there a logical error in my reasoning?

$\endgroup$

1 Answer 1

4
$\begingroup$

If it's significant then the treatment has an effect on the outcome.

This statement is a bit problematic. If the p value is, say 0.049999, this means that if there is actually no association between the the outome and the treatment, then the probability of obtaining this result, or a result more extreme, is 0.049999. However, if the p value was 0.0500001, this would mean that, if there is actually no association between the the outome and the treatment, then the probability of obtaining this result, or a result more extreme, is 0.0500001. Now, if your significance level is 0.05, in the first case you would claim "then the treatment has an effect on the outcome." and in the 2nd case you would not. However, the results are essentially the same. Hence it is better to not rely on p values to make a claim about an "effect" existing.

Also, I would recommend that you avoid using the word "effect" at all as that can often be interpreted by people as a causal claim. What you have found is an association which may or may not be significant at some arbitrary level.

As to the main question, it is a good idea to centre time in this case in order to interpret the intercept more meaningfully:

If I center Time, then I get (-2,-1,0,1,2), so in essence just the mean outcome at the third time point?

Yes., provided that the variable is numeric and not categorical.

But what I need for the level 2 regression to be meaningful is (I think) the mean of the outcome variable across all time points.

This doesn't quite make sense, but it depends on what your other variables are. You say they are "discrete" but does this mean they are, say integers 1, 2, 3, 4 (like the time variable), or are they categorical, such as "blue", "green", "black" ? In the former case, it is the same as time, so the intercept would be the mean of the outcome when the variable is 0, and if this makes sense, you can leave it as that - otherwise centering would be better. However, in the latter case (categorical) then, if contrast coding is used by the software you are using, which is usually the default) the intercept is the mean of the outcome when the categorical variable is at it's reference level, and the estimates for the categorical variable(s) are the difference between the mean of the outcome at each level of the variable, and it's reference level.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.