9
$\begingroup$

This questions might sound stupid, but... is it correct that random effects could apply only to categorical variables (like individual id, population id, ...), e.g. say $x_i$ is categorical variable:

$y_i$ ~ $\beta_{x_i}$

$\beta_{x_i}$ ~ $Norm(\mu, \delta^2)$

but from the principle the random effect cannot apply to continuous variable (like height, mass...), say $z_i$:

$y_i$ ~ $\alpha + \beta \cdot z_{i}$

because then there is only one coefficient $\beta$ which cannot be constrained? Sounds logical but I wonder why it is never mentioned in statistical literature! Thanks!

EDIT: But what if I constrain $z_i$ like $z_i$ ~ $Norm(\mu, \delta^2)$? Is it then random effect? But this is different from the constrain I put on $\beta_{x_i}$ - here I constrain the variable whereas in the previous example I constrained the coefficient! It starts to look as a big mess to me... Anyway, it doesn't make much sense to put this constraint, because $z_i$ are known values, so maybe this idea is completely odd :-)

$\endgroup$
1
6
$\begingroup$

This is a good and a very basic question.

The interpretation of random effects is very domain-specific and is dependent on the modeling choice (the statistical model or being a Bayesian or frequentist). For a very good discussion, see page 245, Gelman and Hill (2007). For a Bayesian everything is random (even though parameters may have a true fixed value, they are modeled as random), and a frequentist can also choose a parameter value to be a fixed effect that would have been otherwise modeled as random (see Casella, 2008, discussion about blocks to be fixed or random in example 3.2).

Edit (after comment)

Data are fixed after you observe them. If they are continuous, they should be modeled as continuous. You can model categorical variables as categorical and sometimes as continuous (like in an ordinal variable setting). The parameters are unknown and they may be modeled as fixed or random. The parameters essentially relate response to predictors. If you want individual predictor's slope (or its coefficient in a linear model) to vary for each response, model it as random, otherwise model it as fixed. Similarly, if you want the intercept to vary regarding groups, then they should be modeled as random; otherwise they should be fixed.

$\endgroup$
6
  • 1
    $\begingroup$ Thanks for your answer, but hmmm.... I've put a simple question and expected somehow simple or general answer :) In your 2nd paragraph you state that in Bayesian everything is random. Please don't complicate it too much :) I intentionally asked at the model level, please forget about frequentist or bayesian - I'm interested in the principle - for which type of variables the random effects are typically used? $\endgroup$ – Tomas Sep 8 '11 at 0:02
  • $\begingroup$ In the 3rd paragraph it you put a special application with time series, I'm not sure I understand it but AFAIK time series are usually discrete (not continuous) if you speak about serial correlation... Anyway, if you have somehow more general answer, please do share. Again, thank you for answer! $\endgroup$ – Tomas Sep 8 '11 at 0:04
  • $\begingroup$ The randomness is in unknown things. When you have observations they are known! The only thing that you know are your data. The unknowns are the parameters and they are may be modeled as random or fixed. (sorry for the confusion; I agree, I unnecessarily made things complicated.) $\endgroup$ – suncoolsu Sep 8 '11 at 0:05
  • $\begingroup$ thanks for simplifying your answer. So coming to the root of my question - so does this mean that the answer is yes, i.e. the random effects apply only to categorical variables, because the random effects apply only to parameters (not data), and the only parameters which can be modelled as random are those with multiple levels, i.e. those of a categorical variable? And the parameters of continuous variables cannot be modelled as random, because there is only 1 parameter value for continuous variable? Thanks! $\endgroup$ – Tomas Sep 8 '11 at 0:26
  • $\begingroup$ In your specific setting, the answer is YES. But in general the answer, I would say, is NO because continuous variables can vary according to groups. For example, income of residents in city should be higher than village (according to you this might be two separate categorical variables for location). $\endgroup$ – suncoolsu Sep 8 '11 at 1:42
3
$\begingroup$

Your question may have already been solved, but it is actually written in a text book;

Random effects are categorical variables whose levels are viewed as a sample from some larger population, as opposite to fixed effects, whose levels are of interest in their own right,

on the page 232 of: Alan Grafen and Rosie Hails (2002) "Modern statistics for the life sciences", Oxford University Press.

$\endgroup$
2
  • 2
    $\begingroup$ This is all true, but I don't think that this answers the question. $\endgroup$ – Patrick Coulombe May 11 '15 at 2:22
  • 1
    $\begingroup$ I disagree with the above comment by @PatrickCoulombe, I think this precisely answers the question. +1. $\endgroup$ – amoeba May 23 '17 at 13:16
1
$\begingroup$

I think the issue is that there are two things involved here. A typical example of random effects might be predicting the grade point average (GPA) of a college student based on a number of factors including their average score in a series of tests during high school.

The average score is continuous. You would typically have a varying intercept, or intercept and slope, for the average score for each individual. The individual is obviously categorical.

So when you say "only applies to categorical variables" it's a little vague. Say you only consider a random intercept for the average score. In this case, your random intercept for a continuous quantity and in fact is probably modeled as something like a gaussian variable with a mean and standard deviation to be determined by the procedure. But this random intercept is determined across a population of students where each student is identified by a categorical variable.

You could use a "continuous" variable instead of student ID. Maybe you could choose a student's height. But it would essentially have to be treated as if it were categorical. If your height measurements were very precise you'd again end up with a unique height for every student so would have accomplished nothing different. If your height measurements were not very precise, you'd end up lumping multiple students together at each height. (Mixing their scores in a possibly ill-defined way.)

This is sort-of the opposite of interactions. In an interaction, you're multiplying two variables and essentially treating both as continuous. A categorical variable would be broken up into a set of 0/1 dummy variables and the 0 or 1 would be multiplied times the other variable in the interaction.

The bottom line is that a "random effect" is in some sense just a coefficient which has a distribution (is modeled) rather than a fixed value.

$\endgroup$
1
  • $\begingroup$ Please log in under your main account to be able to edit. $\endgroup$ – amoeba May 23 '17 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.