Alternatives for random effects anova

Question

We have an experiment that seeks to establish a particular continuous quantity as a possible objective indicator of some subjective categories. Since we cannot share the particular experiment (company confidentiality), here are two other scenarios that I think are similar.

1. Subjective quality of wines such as "full bodied", "fruity", etc. and we imagine that we have measured a chemical that has differing concentrations in the wines that are labeled full bodied versus those that are not. There are a few (say 4) subjective categories, for each of which we obtain 100 samples of wine, and measure the chemical. Finally, the test: Take the mean value of the chemical in each of the four categories, and see if these means are significantly different.
2. Mental disease, going back to a time when certain mental disorders (schizophrenia, a couple others) were diagnosed subjectively. For each of the diseases, we find 100 people who have been diagnosed with that disease, and take an objective measurement (brain chemical or something). Then test: Take the mean value of the chemical for each group of people, and see if these means are significantly different.

The examples are not great. For one, I know nothing about either topic (wine or mental disease). So please understand that these are attempts at examples, and don't focus on them.

In summary, we seek to test if the means of a small number of groups are different, where the members of each group are samples from a larger population.

As a starting point, if there were two categories, a t-test would come to mind (checking that the assumptions of the t-test are ok.)

Here there are N categories (small N), so it's one way anova, which I think would be

model <- lm(Chemical ~ Category, data=Data)

where Category is a column with factor() applied.

And the sampling of the particular 1000 wines or people is a random effect. Correct?

Would this be the model with the random effect?

model <- lmer(Chemical ~ Category + Error(SampleID), data=Data)

EDIT: question is clarified, it had left out the goal of testing for different means! Removed the part about causality, will save that for a separate question.

EDIT: I believe that the basic situation is a test for difference in means between several groups defined by the categories (full-bodied vs fruity vs etc, or schizophrenia vs bipolar vs others), so ANOVA is the starting point. The only change is that there is the random effect resulting from sampling members of each category from a population (e.g. 100 people sampled from the larger population of people with schizophrenia). Without the random effect, the R model would be model <- lm(Chemical ~ Category, data=Data). How can the random effect be specified?

Answer 1

You would only need to include random effects if you had repeated measures, or other clustering of observations. That does not seem to be the case here, and so a simple linear model should work, subject to the usual assumptions:

model <- lm(Chemical ~ Category, data=Data)

Answer 2

The term 'random effect'?

• The sampling error is indeed random, but it is not considered an effect.
• The term 'effect' refers more to something that is correlated or even some direct causal relationship.
• The term 'random effect' is then something that occurs in the relationship as an effect but is in principle random.

Two interpretations of 'random effect'

In a linear model you can describe this for the $i$-th measurement in the $j$-th wine:

$$y_{ij} = \underbrace{a}_{\text{intercept-term}} + \underbrace{\mathbf{x_{ij}} \mathbf{\beta}}_{\text{fixed effect}} + \underbrace{\mathbf{z_{ij}}\mathbf{b}}_{\text{random effect}} + \underbrace{\epsilon_{ij}}_{\text{random error}}$$

This random effect term can be interpreted as a special case of fixed effect or as a special case of error.

• Related to fixed effect: The terms $\mathbf{x_{ij}} \mathbf{\beta}$ and $\mathbf{z_{ij}} \mathbf{b}$ look similar algebraically, the multiplication of a vector with covariates with a vector of coefficients.

  However, the difference is in the assumptions about their distribution. The random effect $\mathbf{b}$ is considered to be sampled from a larger population. This changes the way in which the coefficient(s) $\mathbf{b}$ are estimated. The model has the tendency to place the $\mathbf{b}$ more close to each other (in comparison to OLS) because this has a greater likelihood (at the 'cost' of introducing larger error terms $\epsilon_i$).

  In this view, the random effect is like a fixed effect but with the assumption that is randomly distributed for different $j$.

• Related to error: The 'random effect' can also be incorporated into the random error term (see Intuition about parameter estimation in mixed models).

  You could see the expression $\mathbf{z_{ij}}\mathbf{b}$ as an error term similar to $\epsilon_{ij}$. But, when the $z_{ij}$ have the same value of the subscript $j$ 8when they are the same wine), the difference is that the same error (the random effect part of it) occurs. (this repetition of the same error is very useful in allowing to filter it out during the analysis, see Paired difference tests for a simple case.).

  If you incorporate the error term $\mathbf{z_{ij}}\mathbf{b}$ and the error term $\epsilon_{ij}$ into a single error term then the error for different $ij$ is not anymore independent but you get correlated error terms when the subscript $j$ is the same.

Illustration

In the image below, you see an example for sampling wines (using only one covariate term 'full-bodiedness', but this $x$-axis may be considered in to be multi-dimensional).

On the right image you see a big oval shape (created by simulation) which represents a relationship between 'full bodiedness' and 'chemical'. On the left image you see that this relationship can be split up and seen as a sum of two errors (in theory you might split this up further, the approximate Gaussian distribution of the error term may be seen as the average sum of many little errors). (1) one source of error stems from the specific selected wine (e.g. a selected brand or a selected region). (2) another source of error stems from errors that occur within the same wine.

If you would have multiple measurements within a same wine then you have the left image, otherwise you have the right image. Note: the distribution, underlying model did not change (the same oval shape in both images), but it is about how you perform the measurements/experiment/sampling.

So, if you only have single measurements for your SampleID (which is random) then considering it as a 'random-effect' is not very useful. (1) There is no correlation (effect) to calculate, between 'SampleID' and 'chemical', when you only have a single measurement point for each single SampleID (this is a practical view, it does not mean that this correlation may not exists). (2) There is no correlation between the errors because all the $\mathbf{z_{ij}}\mathbf{b}$ are independent (there is no repetition, dependence, in $\mathbf{z_{ij}}$ if this is considered to be SampleID without repetition).

^{This is the basic consideration... Q:Do you repeat a measurement within the same class for which you may consider a random effect that occurs repeatedly? After this consideration you may wonder about additional things like whether your random effect is crossed vs nested}