8
$\begingroup$

Now I'm having a hard time having a grasp on the difference between fixed and random effects of regression models. I believe I understand it's recommended to use random effects if you consider heterogeneity of slopes, when the data is nested among hierarchical levels, etc.

But here's the question.

  1. Why don't we just put moderating variable(interaction term) if we want to reflect the changing effect among different groups? for example, if the effect of study time on GPA differs among different classrooms, then why not just make a dummy variable for classroom variable, and put an interaction term? I cannot understand what the point is here.

  2. What is an overall intuition on the grand assumption of random effects model? what is the main idea that can penetrate the logic of random effects model? I don't want any mathematical or statistical explanation, I want to draw some hypothetical picture in my head.

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ About the dummy variable, that works if the variable has a limited number of values (like classrooms in your case) but not when there are a hughe number of values and that is the trick; if you have a hughe number of values then you get a hughe number of intercepts (or slopes) thus a lot of dummies and then you can not estimate the model well (you loose many degrees of freedoms because you have a lot of explanatory variables). In that case you can use random effects; i.e. you assume that the intercepts are normally distributed and then your hughe number of dummies ... $\endgroup$
    – user83346
    Oct 2, 2016 at 8:59
  • 3
    $\begingroup$ ... is ''summarised'' in a normal distribution. The latter has only two parameters (mean and standard deviation), so in stead of estimateing a hughe number of corefficients (namely one for each your dummies) you only have to estimate two parameters (mean and standard deviation) and you know the distribution of the intercepts. This saves a lot of degrees of freedom. $\endgroup$
    – user83346
    Oct 2, 2016 at 9:01
  • $\begingroup$ wow. thanks it really helped. So many people gave explanations in an inconsistent way, but I think this summarized everything up. $\endgroup$
    – Kang Inkyu
    Oct 2, 2016 at 9:37
  • $\begingroup$ @fcop one more question, does your explanation have something to do with clustering effect(or nested effect if you will)? I cannot find an intuitive link between them. Is your comment appliable to the logic of clustering? Sorry for my bad English by the way. $\endgroup$
    – Kang Inkyu
    Oct 4, 2016 at 12:23

3 Answers 3

Reset to default
13
$\begingroup$

One way to think about fixed-effects vs. random effects is by examining how the fixed-effects estimator works in comparison to the random effects estimator.

Let's say I have panel data on firms. Let $y_{i,t}$ be dividends for firm $i$ at time $t$. Let $x_{i,t}$ be something we're looking at like free cash flow.

Imagine our model is:

$$ y_{i,t} = \beta x_{i,t} + u_i + \epsilon_{i,t} $$

So dividends for firm $i$ at time $t$ are the sum of $\beta$ times free cash flow plus a firm specific effect $u_i$ and a firm, time specific error-term $\epsilon_{i,t}$. Now let's imagine two different estimators:

  • The within estimator. $\beta$ is estimated using only time-series variation within each firm.
  • The between estimator. $\beta$ is estimated using only the variation between different firms. (The between estimator is $\beta$ from the cross-sectional regression $\bar{y}_i = \beta \bar{x}_i + v_i$.)

The within estimator is the fixed-effect estimator. It takes off the mean from each group and the only variation leftover to estimate $\beta$ is time series variation within each firm. If the fixed effects can be anything, this is what you have to do.

The random effects estimator is a weighted average of the within estimator and the between estimator. If the effects $u_i$ are random and mean zero, then variation between firms also contains information about $\beta$ and the between estimator is also a consistent estimator. Rather than tossing out the between firm variation (as occurs in the fixed effect estimator), the between firm variation is given some weight in the random effects estimator of $\beta$.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Rather than tossing out the between firm variation (as occurs in the fixed effect estimator), the between firm variation is given some weight in the random effects estimator of 𝛽. I could create a categorical binary matrix that encodes the firm's identity and plug this matrix side-by-side with x, and estimate it as if it is a fixed-effect to obtain firm-specific parameter estimates using good old least squares, right? How this approach differs from the random effects approach you explain, doesnt it gimme the same info? $\endgroup$
    – bonobo
    May 20, 2022 at 18:53
6
$\begingroup$

You can start with this thread. As already noted in comments by fcop one example of using random effects is then you have multiple levels of your variable (classrooms) and estimating so many parameters would require large amounts of data and huge computational power. In such cases often you wouldn't be interested in classroom effects themselves, but their influence in general, you would assume that they vary but can be summarized using common distribution. It also could be the case that you have just a sample of classrooms and the particular classrooms are not interesting by themselves, but are used to learn something about general variability that is connected with classrooms. So you use random effects what you are not interested in estimating the parameters for your variable precisely, yet you want to account for influence of such variable by estimating the distribution of possible influences of it's levels.

Improve this answer
$\endgroup$
13
  • $\begingroup$ Thanks a lot. I wanna ask one more additional question: $\endgroup$
    – Kang Inkyu
    Oct 2, 2016 at 9:38
  • $\begingroup$ Say I wanna analyze a dataset of 100 samples. If I create a dummy that represents each sample(say the name of the variable is Individual Nature, IN) then I would need 99 dummy variables. But I'm not really interested in the individual effects themselves(as many common analysis are) but I want a summarized common distribution, which means I wanna consider all people of sample about the same. If this is the case, can I call it random effects model? or is this a fixed effects model because I only have 1 level of data? $\endgroup$
    – Kang Inkyu
    Oct 2, 2016 at 9:44
  • 1
    $\begingroup$ Oh, I see what you mean. I think I was confused it from individual fixed vs individual random effects of panel data. Panel data contains a couple of samples within each individual, so in that case I think it makes sense to make a random effects model considering each individual sample as a group, right? Thanks for the comment. Now I'm getting closer and closer to the ultimate grasp. $\endgroup$
    – Kang Inkyu
    Oct 2, 2016 at 11:01
  • 1
    $\begingroup$ @KangInkyu answering shortly: yes it does. You can find clear introduction in this book stat.columbia.edu/~gelman/arm $\endgroup$
    – Tim ♦
    Oct 4, 2016 at 12:26
  • 1
    $\begingroup$ @KangInkyu example: if classes are nested within schools, then they have something in common, they come from similar distribution -- this is what nesting is about. $\endgroup$
    – Tim ♦
    Oct 4, 2016 at 12:39
5
$\begingroup$

About the dummy variable, that works if the variable has a limited number of values (like classrooms in your case) but not when there are a hughe number of values and that is the trick; if you have a hughe number of values then you get a hughe number of intercepts (or slopes) thus a lot of dummies and then you can not estimate the model well (you loose many degrees of freedoms because you have a lot of explanatory variables).

In that case you can use random effects; i.e. you assume that the intercepts are normally distributed and then your hughe number of dummies is ''summarised'' in a normal distribution. The latter has only two parameters (mean and standard deviation), so in stead of estimateing a hughe number of corefficients (namely one for each your dummies) you only have to estimate two parameters (mean and standard deviation) and you know the distribution of the intercepts. This saves a lot of degrees of freedom.

Improve this answer
$\endgroup$
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.