0
$\begingroup$

My problem is the following:

I have 1/0 "conversion" outcomes.

I have samples from three groups: people who speak russian, people who speak japanese, people who speak portuguese. i expect the conversion rate will be different from each group.

each language group is placed within a treatment or control cohort randomly. there is a 50/50 split within each language group.

so i have one coefficient, $T$, which is a fixed effect, for the treatment effect.

then i have one other coefficient, $L$, which I think should be the random effect for each group.

$Y_i = \alpha + T_i + L_{\gamma} + e$

where

$L_{\gamma} = N(0, \sigma_\gamma^2 I)$

i believe this is correct because I expect the conversion (and therefore the variance) to vary by group. the people sampled for each language group are also a random sample. therefore, to stabilize the variance, it seems to be that we must have an error term, and a random effect, for each language group. a fixed categorical effect it would seem to me is not quite right because the assumption would be equal variance for each sample.

does this seem right?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

This does not seem right to me. It seems like you want to put a random effect of language, which means that language-specific intercepts are assumed to have a mean and variance. But there are only three languages; how do you think you can estimate the variance of the random effect with only three levels?

Random effects are used when you have multiple observations per individual, in which case there is a random effect for the individual intercepts (and possible slopes). In your design, you did not suggest each individual is observed more than once. So there is no way to specify a random effect for individual.

The model you should be using is a very simple one: $$ Y \sim 1 + T + L + T \times L $$ You can fit this using ordinary least squares and use a robust standard error; there is no misspecification because the model is fully saturated. This is a very simple 2x3 design, or a one-way design with a 3-category moderator. I think you might be overthinking it if you think a random effect is necessary, unless you have not described the element of the design that would necessitate a random effect.

Improve this answer
$\endgroup$
7
  • $\begingroup$ OK I see. Despite reading it many times I continue to struggle with the random/fixed effect distinction. I thought random effect simply meant: The effect has non-homogenous variance and needs to be stabilized. Hence my reasoning for why I thought it needed its own error term. But i see now that it's more about having multiple observations within a cluster or a block of some kind, such as repeated individual measurements. $\endgroup$
    – Estimate the estimators
    Sep 26 at 16:41
  • $\begingroup$ One q for you: DO you have any resource recommendations on knowing when to use clustered/sandwich standard errors vs. random effects? It seems very similar to me. $\endgroup$
    – Estimate the estimators
    Sep 26 at 16:47
  • $\begingroup$ Sure: doi.org/10.1037/met0000078 and doi.org/10.1080/00273171.2016.1167008. For generalized linear models the choice matters more; for linear models, not so much. It depends on the goal of your analysis and the assumptions you are willing to make. $\endgroup$
    – Noah
    Sep 26 at 16:56
  • $\begingroup$ you are very helpful. These resources give me a lot to learn from. $\endgroup$
    – Estimate the estimators
    Sep 26 at 17:53
  • $\begingroup$ One question: Why is the model saturated? We have many data points per language . Was that part not clear? We have, say, 1k points per language. I thought saturated meant n(X) = n(coefficients) $\endgroup$
    – Estimate the estimators
    Sep 27 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.