4
$\begingroup$

I am used to seeing the linear mixed-effects model in the form: $$Y_{ij} = \mu + \alpha_i + \gamma_j + (\alpha\gamma)_{ij} + \epsilon_{ij}$$ assuming $ \gamma_j i.i.d \sim N(0,\sigma^2_{\gamma})$ and $ (\alpha\gamma)_{ij} i.i.d \sim N(0,\sigma^2_{\alpha\gamma})$

where $\alpha_i$ is the fixed effect of machine $i$ and $\gamma_j$ is the random effect of worker $j$ and $(\alpha\gamma)_{ij}$ is the corresponding random interaction. And assuming the data looks like below:

## Classes 'nffGroupedData', 'nfGroupedData', 'groupedData' and 'data.frame':   54 obs. of  3 variables:
##  $ Worker: Factor w/ 6 levels "1","2","3","4",..: 1 1 1 2 2 2 3 3 3 4 ...
##  $ Machine: Factor w/ 3 levels "A","B","C": 1 1 1 1 1 1 1 1 1 1 ...
##  $ y  : num  52 52.8 53.1 51.8 52.8 53.1 60 60.2 58.4 51.1 ...

and also I see ANOVA in the following form:

$$Y_{ij} = \mu + \alpha_i + \epsilon_{ij}$$

where $\alpha_i$ is known as the treatment effect and $\epsilon_{ij}\;iid\;\sim N(0,\sigma^2)$

I start to ask myself why is it that there are no covariates $x_{ij}$ anywhere in the both model, only the coefficients are there. Clearly although Worker and Machine are factors, they are still covariates and hence should appear as $x_{ij}$ somewhere in the model. I am sensing that I might have misunderstood something here. Why are the covariate terms suppressed in the model at the top ? How do I reconcile the notations in this case with linear regression model $Y_i = \beta_0 + \beta_1 X_1 + \epsilon_i$?

Please guide me here. Thanks.

$\endgroup$
7
  • $\begingroup$ The $x_{ij}$ you are looking for are dummy variables which indicate to which worker and machine a given observation is associated. $\endgroup$ Aug 13, 2020 at 5:40
  • 1
    $\begingroup$ Why do you have a $k$ on $\epsilon$ but none on $y$? $\endgroup$ Aug 13, 2020 at 8:18
  • 1
    $\begingroup$ @JesperforPresident I just noticed that too. It's obviously a typo (which I've just fixed). $\endgroup$ Aug 13, 2020 at 8:19
  • 1
    $\begingroup$ @JesperforPresident Yes it looks like a balanced fatorial experiment to me, though I can't speak for the OP of course ! $\endgroup$ Aug 13, 2020 at 8:23
  • 1
    $\begingroup$ In this note www2.compute.dtu.dk/courses/02429/enotepdfs/eNote-4.pdf there are some examples where the covarites (dummyvariables) for the coefficients are wirtten explicitly thus answering the part of the question: I start to ask myself why is it that there are no covariates? $\endgroup$ Aug 13, 2020 at 9:20

1 Answer 1

3
$\begingroup$

I can understand your confusion. It really all depends on how you define your notation, and what background you have in these topics.

Models like this stem from 4 different strands of methodological research: hierarchical linear models (eg Bryk & Raudenbush, 1992), multilevel models (eg Snijders & Bosker, 2012) and mixed effects models (eg Pinheiro & Bates, 2000), and the analysis of planned experiments (eg ANOVA, random effects ANOVA, factorial experimens etc). The first two are very similar, but the first evolved mostly in the USA, whereas the 2nd evolved mostly in europe; the third is fairly ubiquitous, while the 4th can be thought of as a class of more general models of which the first thre are specific examples.

The notation of your first model is a textbook example of BOTH a two-factor factorial experiment (with two random factors) AND a two-factor mixed effects model (with one random factor)

$$Y_{ij} = \mu + \alpha_i + \gamma_j + (\alpha\gamma)_{ij} + \epsilon_{ij}$$

This model says we have 2 factors, machine and worker in your case, where 3, and 6 machines are workers, respecively, have been sampled from a wider population of such machines and workers. This would then be a variance components model. Of course the usual objections to the number of levels of both factors would apply. Alternatively (and this is the way you using it) we can say that machine is a fixed effect, and then we have a mixed effects model.

So we can see that using this notation, we are using the subsripts to denote the levels of each factor:

$$ \left\{\begin{array}{c} \begin{align} i&=1,2,3 \\ j&=1,2, \dots,6 \\ \end{align} \end{array}\right. $$

That is the usual type of notation in experimental design analysis, hence the similar notation in your ANOVA model: $$ Y_{ij} = \mu + \alpha_i + \epsilon_{ij} $$

However, in the last model you wrote:

$$ Y_i = \beta_0 + \beta_1 X_1 + \epsilon_i $$

the assumption is that $X_1$ is the name of continuous variable, not a level 1 of a factor called $X$ . If it were a factor then we could use the notation above, or we could create seperate dummy variables.

$\endgroup$
3
  • $\begingroup$ so we can also write $Y_{ij} = \mu + \alpha_i Z_1 + \gamma_j Z_2 + (\alpha_i\gamma_j Z_1 Z_2) + \epsilon_{ij}$ just to acknowledge the existence of the factors $Z_1, Z_2$ , right ? I know this is unorthodox. But this notation also makes sense , correct ? $\endgroup$ Aug 13, 2020 at 9:19
  • $\begingroup$ With notation you always need to define what everything is. In this case, are $Z_1$ and $Z_2$ different levels of the same factor ? You can defie a model any way you wish, provided that it makes sense mathematically. In this case I'm not sure what the $\alpha_i\gamma_j Z_1 Z_2$ term is for. Is it an interaction ? Anyway, please don't ask follow-up questions in comments. This is worthy of a new question, just make sure you explain exactly what all the terms, coefficients and subecripts are. $\endgroup$ Aug 13, 2020 at 9:25
  • $\begingroup$ It seems to me you are wondering what the model matrix looks. As @Robert Long comments this involves creating dummy variables. See for example the note posted above. $\endgroup$ Aug 13, 2020 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.