# Multi-Level Model with two scores per level 2 unit - reasonable analysis?

I have an experimental design with attitudes toward one positive and one negative stimulus nested within individuals. I also have a continuous predictor at the person level (a personality construct).
My plan was now to build a multi-level model with valence as level-1 predictor, (centered) personality as level-2 predictor, and the cross-level interaction of these two variables. Since I wanted to use nested chi-square statistics to assess the individual effects, the code would be something like this:

    library(nlme)
mod0 <- lme(attitude~ 1, random = ~1|ID, data=dat, method="ML")

mod1 <- lme(attitude ~ valence, random = ~valence|ID, data=dat, method="ML")

mod2 <- lme(attitude ~ valence+z_personality, random = ~valence|ID, data=dat, method="ML")

mod3 <- lme(attitude ~ valence*z_personality, random = ~valence|ID, data=dat, method="ML")


My questions are the following:

1) Is it justified to use multi-level models, given that I have only two observations per participant?

2) The random variance for valence is exaclty defined (with only two data points per person, there are no degrees of freedom left; the standard error for the random variance estimate is 0). Should I include a random effect for valence in this case?

3) I am particularly interested in the cross-level interaction (dependeing on personality, some participants are hypothesized to have a more positve attitude toward the negative stimulus than toward the positive stimulus). If I do not include the random variance for valence (see 2), this - in my understanding - means that the difference between positive and negative stimuli is the same for all participants. However, this is explicitly not what I expect. To put the question simply: Do I need to specify random variance for a level-1 predictor if I am interested in the cross-level interaction of this predictor?

For all these points the question is not "Can R / SPSS do this?" (I have tried, both can do it), but rather if I can reasonably interpret the results, given my design. Also, if you had some references for me to back this up, this would be greatly appreciated.

Thanks for your help!

## 1 Answer

I'll answer your second question first. Yes, there is no variance within the subject level in your situation, so including a random slope makes no difference when you run your analyses. In fact, in your situation, a multi-level model is exactly equivalent to a regression analysis with a difference score as the dependent variable (which, in turn, is equivalent to an ANOVA model with a 2-level within-subjects factor and an individual difference variable as a covariate that is allowed to interact with the within-subjects factor).

To demonstrate this, I have simulated some data and analyzed those data using both a multi-level model (using lmer from the lme4 package) and a regression model (using lm).

require(lme4)
require(reshape2)
require(plyr)

# Set the seed
set.seed(12315)

# Simulate data
id <- rep(1:100, 2)
valence <- c(rep(-.5, 100), rep(.5, 100))
personality <- rep(rnorm(100, sd = 1))
eval <- .5 * valence * personality
d_long <- data.frame(id, valence, personality, eval)
d_long <- ddply(d_long, "id", mutate,
int = rnorm(1, sd = .5),
val_b = rnorm(1, sd = .5),
eval = eval + int + val_b * valence)
d_long$eval <- d_long$eval + rnorm(200, sd = .5)
d_long <- d_long[, c("id", "personality", "valence", "eval")]

# Fit the multi-level model
mod <- lmer(eval ~ valence * personality + (1 | id), data = d_long)
summary(mod)

Linear mixed model fit by REML ['lmerMod']
Formula: eval ~ valence * personality + (valence | id)
Data: d_long

REML criterion at convergence: 725.5223

Random effects:
Groups   Name        Variance Std.Dev. Corr
id       (Intercept) 1.045    1.022
valence     1.156    1.075    0.53
Residual             1.032    1.016
Number of obs: 200, groups: id, 100

Fixed effects:
Estimate Std. Error t value
(Intercept)         -0.27116    0.12497  -2.170
valence             -0.01701    0.17949  -0.095
personality         -0.03500    0.13659  -0.256
valence:personality  0.63373    0.19617   3.231

Correlation of Fixed Effects:
(Intr) valenc prsnlt
valence      0.262
personality -0.029 -0.008
vlnc:prsnlt -0.008 -0.029  0.262


Compare the results of the multi-level model to those with lm using the same data that I reshaped into so-called "wide" format so that the data would be compatible with lm:

# Reshape the data into wide format for analysis using lm
d_wide <- dcast(d_long, id + personality ~ valence, value.var = "eval")
colnames(d_wide) <- c("id", "personality", "low_eval", "high_eval")

# Predict the difference between high_eval and low_eval from personality
mod <- lm(high_eval - low_eval ~ personality, data = d_wide)
summary(mod)

Call:
lm(formula = high_eval - low_eval ~ personality, data = d_wide)

Residuals:
Min      1Q  Median      3Q     Max
-4.7237 -1.1407 -0.1562  1.1338  5.4332

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.01701    0.17948  -0.095  0.92469
personality  0.63373    0.19617   3.231  0.00168 **
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.794 on 98 degrees of freedom
Multiple R-squared: 0.09624,    Adjusted R-squared: 0.08702
F-statistic: 10.44 on 1 and 98 DF,  p-value: 0.001683


Note that I obtained precisely the same coefficient value and $t$-value for the personality by valence interaction using lmer and lm. Although the analyses using lm that I show above do not include the main effect of personality, you can also obtain this effect through lm by predicting the average of high_eval and low_eval from personality. In general, when you only have two levels to your within-subjects variable, you can replicate the coefficient values and $t$-values from a multi-level model using a standard linear model (although this is not necessarily the case for the degrees of freedom and $p$-values; see a discussion by Doug Bates here).

This, I think, answers your second and third questions. There's nothing wrong necessarily with using a multi-level model in your situation, but you don't really gain anything in using it relative to a plain regression analysis of difference scores / averages. Likewise, there's nothing that prevents you from testing interactions between some individual difference variable and your within-subjects manipulation. What multi-level models allow (when you have more than two measurements per participant) is for random variation within multiple levels -- the interaction between the individual difference and the within-subjects manipulation is a fixed effect.

• This is a helpful answer (+1) but be a bit careful with you coding example: 1. You need plyr to use the function ddply(), 2. you call ddply on a dataframe that doesn't exist ($d$). On a different note: while you correct say that you get the same $t$-values, you need to draw attention to the fact that defining the related degrees of freedom for that $t$-distribution is not a well defined aspect and people should not be tempted to naively extract $p$-values from it. Same $t$-values do not lead to same $p$-values in this case. (Nice answer nevertheless) – usεr11852 Jan 17 '14 at 1:38
• You are quite right, of course. I have made corrections to both the code of my answer and have made my claims more precise (and linked to the discussion by Doug Bates about $df$s and $p$-values in multi-level models). – Patrick S. Forscher Jan 17 '14 at 16:04