In a lab experiment Participants took part in four different treatment. At the beginning of each session they were randomly assigned to groups of 4 and played repeated public goods game for 30 periods.

So far I have collapsed the data to obtain group averages, so that observation in a period are independent. This data was used to estimate a model with group-level random effects. Since in some groups there were participants whose incentives were different than those of other members of their group, I would like to test whether difference in incentives lead to disparities in behaviour.

Would it be OK for me to split the group average into two observations:

  • With the decisions made by the "special" individuals
  • with the average of 3 others

The same random effect would be assigned to both.

  • $\begingroup$ It seems like you want to use a mixed effect regression model here, without performing any preemptive collapsing of the data into averages. Then the fixed effect would be an indicator for "special" individuals and the group-level averages would be a random effect. If that's not clear I'm happy to flesh out more as a full-fledged answer $\endgroup$ – Michael Oberst Dec 29 '16 at 16:20
  • $\begingroup$ Thanks for your comment. "group-level averages would be a random effect" - do you mean the groups' means of all 30 rounds? Also, my concern is that without collapsing the data, how can I take into account the fact that the earnings of each individual are correlated, ie endogenous to the group. PS I used the following command in Stata to estimate the model using the group averages as observations: xtmixed Profit Treatment Period Treatment#(c.Period ) || groupintreatment: , mle residuals(independent, by(Treatment)) $\endgroup$ – user3771956 Dec 29 '16 at 16:46
  • $\begingroup$ Ah I overlooked the note about 30 rounds - you are correct that you have to account for that. I'm less familiar with stata, but could lay out the formula in R (using lme4) if that's helpful $\endgroup$ – Michael Oberst Dec 29 '16 at 16:54
  • $\begingroup$ I'm using R, but used Stata after not being able to find a proper solution in R. Could you please post the formula? $\endgroup$ – user3771956 Dec 29 '16 at 17:07
  • $\begingroup$ Also, if you could refer me to some reading material on the topic that would be great. Thanks $\endgroup$ – user3771956 Dec 29 '16 at 17:09

Short answer & R Formula

I believe the R code you're looking for is something along the lines of the following, from the lme4 package.

myModel <- lmer(y ~ incentive + (1 | treatGroup/player), data = df)

Basically you're saying that there's a random effect for each player, who is nested inside of a treatment group with a random effect. In addition, there is a fixed effect for incentive.

Longer answer / explanation

First of all, I personally find the terms "fixed effect" and "random effect" to be confusing and used somewhat inconsistently. Andrew Gelman (Prof. at Columbia) has a great blog post on the topic.

Model formulation

So in lieu of using those terms, I'll try to write out the (basic) formulation based on what I understand from your question. In particular, it sounds like we have:

  • 16 players (indexed by $i$)
  • 30 observations per player (indexed by $j$)
  • 4 possible groups (indexed by $k$)
  • A binary "incentive" treatment $M$, where $m_{i(k)} = 1$ if the $i$-th player receives the incentive

Let $Y_{ij(k)}$ be the $j$-th observation for the $i$-th player, who is placed into the $k$-th group.

With these in mind, our regression model is as follows:

$$Y_{ij(k)} \sim N(\mu_{i(k)}, \sigma^2)$$

$$\mu_{i(k)} = \alpha_i + \alpha_k + \beta \cdot m_{i(k)}$$

Explanation of terms / quick notes

We have a few relevant terms:

  • $\alpha_i$: The 'baseline' for this player
  • $\alpha_k$: the effect of being in the $k$-th group
  • $\beta$: The impact of the incentive treatement

Some quick notes:

  • Using (1 | treatGroup / player) describes a nesting between levels (e.g., each player is in only one treatment group).

  • To your point in the comments, 30 observations for each player are assumed to be drawn from a common distribution - Our goal is to estimate the impact of (a) incentives and (b) treatment group on the mean of that distribution.

  • This answer assumes no interaction between the incentive and the treatment group.

Additional reading

  • I used this to refresh my memory on lmer syntax since I'm more accustomed to using rstan (link).

  • Skimming this tutorial (I haven't read the whole thing), it looks like a good resource, with an intuitive overview for the beginner!

  • Also see this CV post on lmer formula syntax

If you really want to get into this stuff, then Gelman & Hill (2006) is an excellent book.

  • $\begingroup$ Thanks again for the answer. Since I have multiple treatments, can I add a "fixed effect" if I wish to compare between them? I need to tell you that the allocation to the treatments is random, as well as allocation to a group with in the treatment. $\endgroup$ – user3771956 Jan 1 '17 at 22:43
  • $\begingroup$ And another question (hopefully the last one). If I need p-values (I've seen that people suggest that we should abandon them, but I till need them), would you recommend lmerTest for my case. $\endgroup$ – user3771956 Jan 1 '17 at 23:16
  • $\begingroup$ or maybe with mixed function from the afex package? $\endgroup$ – user3771956 Jan 1 '17 at 23:34
  • 1
    $\begingroup$ Not sure about measuring differences between two different treatments - I'd recommend posting as a separate question. Same to your question about pvalues - I think (?) you should be able to use the pvalues for the treatment coefficients in the regression model but might be worth fleshing out exactly what kind of hypothesis you are trying to test and asking as a distinct question - I use multilevel models a lot but not so much in an experimentation context $\endgroup$ – Michael Oberst Jan 2 '17 at 1:04
  • $\begingroup$ Thank you again, I might add another question. On top of what was asked in the my question, what would you do if the selection to the incentive is endogenous? I think this renders multi-level approach to be inappropriate as there might be correlation between some unobserved factors and the incentive. Would removal of individuals' effects solve this problem? $\endgroup$ – user3771956 Jan 2 '17 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.