We have an experiment that seeks to establish a particular continuous quantity as a possible objective indicator of some subjective categories. Since we cannot share the particular experiment (company confidentiality), here are two other scenarios that I think are similar.

  1. Subjective quality of wines such as "full bodied", "fruity", etc. and we imagine that we have measured a chemical that has differing concentrations in the wines that are labeled full bodied versus those that are not. There are a few (say 4) subjective categories, for each of which we obtain 100 samples of wine, and measure the chemical. Finally, the test: Take the mean value of the chemical in each of the four categories, and see if these means are significantly different.
  2. Mental disease, going back to a time when certain mental disorders (schizophrenia, a couple others) were diagnosed subjectively. For each of the diseases, we find 100 people who have been diagnosed with that disease, and take an objective measurement (brain chemical or something). Then test: Take the mean value of the chemical for each group of people, and see if these means are significantly different.

The examples are not great. For one, I know nothing about either topic (wine or mental disease). So please understand that these are attempts at examples, and don't focus on them.

In summary, we seek to test if the means of a small number of groups are different, where the members of each group are samples from a larger population.

As a starting point, if there were two categories, a t-test would come to mind (checking that the assumptions of the t-test are ok.)

Here there are N categories (small N), so it's one way anova, which I think would be

model <- lm(Chemical ~ Category, data=Data)

where Category is a column with factor() applied.

And the sampling of the particular 1000 wines or people is a random effect. Correct?

Would this be the model with the random effect?

model <- lmer(Chemical ~ Category + Error(SampleID), data=Data)

EDIT: question is clarified, it had left out the goal of testing for different means! Removed the part about causality, will save that for a separate question.

EDIT: I believe that the basic situation is a test for difference in means between several groups defined by the categories (full-bodied vs fruity vs etc, or schizophrenia vs bipolar vs others), so ANOVA is the starting point. The only change is that there is the random effect resulting from sampling members of each category from a population (e.g. 100 people sampled from the larger population of people with schizophrenia). Without the random effect, the R model would be model <- lm(Chemical ~ Category, data=Data). How can the random effect be specified?

  • $\begingroup$ Perhaps it should be split into two questions, one about the Category~Chemical versus Chemical~Category, and the second about how to express it in R. I was hoping that the first question would be easy to dismiss and we could get to the second question. $\endgroup$ – sportscan Jun 26 '18 at 6:16
  • $\begingroup$ You might want to go trough this : en.wikipedia.org/wiki/Logistic_regression which is a classical example of modeling a categorical variable as function of a continuous variable. But it is a basic example. If you could specify your question more. Ie. not necessarily leaving out the R-codes, but more about what you want, prediction versus inference, point estimate or estimate confidence intervals, give information about the data collection/population from which it is drawn and process, etc. Why (what for) are you using the model? $\endgroup$ – Sextus Empiricus Jun 26 '18 at 7:37
  • $\begingroup$ I edited the question to clarify. Indeed, it had left out the goal, which is to test for difference of means of the groups. I have worked though a tutorial on logistic regression (Ng's), which was clear. Logistic regression conventionally gives a classification accuracy, but in this case we want something like a t-test for difference of means. However, maybe it is a good idea, it gives an alternate statement. $\endgroup$ – sportscan Jun 26 '18 at 18:27
  • $\begingroup$ It still sounds like you don't need random effects, and that a plain old linear model (1-way ANOVA) will do what you want ... $\endgroup$ – Ben Bolker Jun 28 '18 at 13:38
  • $\begingroup$ The categories are fixed $\endgroup$ – sportscan Jul 1 '18 at 17:26

You would only need to include random effects if you had repeated measures, or other clustering of observations. That does not seem to be the case here, and so a simple linear model should work, subject to the usual assumptions:

model <- lm(Chemical ~ Category, data=Data)
  • $\begingroup$ Several textbooks indicate a random effect should be used when the selecte d individuals are a subset of a larger population. In the book Dehlert A First Course in Design and Analysis of Experiments, p. 254, discussing an experiment in which a company has 50 machines and 10 are selected for texting. "These fixed-effects models are not appropriate for our carton strength data. .... the fixed-effects assumptions don't make much sense here for a couple of reasons. First, we are trying to learn about and make inferences about the whole population of machines, $\endgroup$ – sportscan Jun 29 '18 at 5:40
  • $\begingroup$ (continued) not just these ten machines that we tested in the experiment, so we need to be able to make statements for the whole population, not just the random sample that we used in the experiment." $\endgroup$ – sportscan Jun 29 '18 at 5:41
  • $\begingroup$ @sportscan that sounds very much like an experiment where there are multiple measures on the same machines. The machines are assumed to be drawn from a larger population of machines, and you want to be able to make inferences that are generalisable to the whole population of machines. This is part of the classic definition of the situation where you do need random effects - because you have multiple measures/clustered data. The description of your experiment is not like that, so you don't need random effects. $\endgroup$ – Robert Long Jun 29 '18 at 7:19
  • $\begingroup$ In the same way that the machines are drawn from a larger population of machines, the wines are drawn from the population of wines, and the 100 people with the particular (subjectively categorized) disease are drawn from the larger population of such people. Is it not the same situation? $\endgroup$ – sportscan Jun 30 '18 at 1:41

The term 'random effect'?

  • The sampling error is indeed random, but it is not considered an effect.
  • The term 'effect' refers more to something that is correlated or even some direct causal relationship.
  • The term 'random effect' is then something that occurs in the relationship as an effect but is in principle random.

Two interpretations of 'random effect'

In a linear model you can describe this as:

$$y_i = \underbrace{a}_{\text{intercept-term}} + \underbrace{\mathbf{x_i} \mathbf{\beta}}_{\text{fixed effect}} + \underbrace{\mathbf{z_i}\mathbf{b}}_{\text{random effect}} + \underbrace{\epsilon_i}_{\text{random error}}$$

This random effect term can be interpreted as a special case of fixed effect or as a special case of error.

  • Related to fixed effect: The terms $\mathbf{x_i} \mathbf{\beta}$ and $\mathbf{z_i} \mathbf{b}$ look similar algebraically, the multiplication of a vector with covariates with a vector of coefficients.

    However, the difference is in the assumptions about their distribution. The random effect term $\mathbf{z_i}\mathbf{b}$ is considered to be sampled from a larger population. This changes the way in which the coefficient(s) $\mathbf{b}$ is estimated. The model has the tendency to place the term $\mathbf{z_i}\mathbf{b}$ more close to each other because this has a greater likelihood (at the 'cost' of introducing larger error terms $\epsilon_i$).

    In this view the random effect is like a fixed effect but with the assumption that it is randomly distributed.

  • Related to error: The 'random effect' can also be incorporated into the random error term (see Intuition about parameter estimation in mixed models).

    You could see the expression $\mathbf{z_i}\mathbf{b}$ as an error term similar to $\epsilon_i$. But, when the $z_i$ occurs multiple times, the difference is that the same error (the random effect part of it) occurs multiple times! (this repetition of the same error is very useful in allowing to filter it out during the analysis, see Paired difference tests for a simple case.).

    If you incorporate the error term $\mathbf{z_i}\mathbf{b}$ and the error term $\epsilon_i$ into a single error term then the error for different $i$ is not anymore independent but you get correlated error terms.


In the below image you see an example for sampling wines (using only one covariate term 'full-bodiedness', but this $x$-axis may be considered in to be multi-dimensional).

On the right image you see a big oval shape (created by simulation) which represents a relationship between 'full bodiedness' and 'chemical'. On the left image you see that this relationship can be split up and seen as a sum of two errors (in theory you might split this up further, the approximate Gaussian distribution of the error term may be seen as the average sum of many little errors). (1) one source of error stems from the specific selected wine (e.g. a selected brand or a selected region). (2) another source of error stems from errors that occur within the same wine.

If you would have multiple measurements within a same wine then you have the left image, otherwise you have the right image. Note: the distribution, underlying model did not change (the same oval shape in both images), but it is about how you perform the measurements/experiment/sampling.

So, if you only have single measurements for your SampleID (which is random) then considering it as a 'random-effect' is not very useful. (1) There is no correlation (effect) to calculate, between 'SampleID' and 'chemical', when you only have a single measurement point for each single SampleID (this is a practical view, it does not mean that this correlation may not exists). (2) There is no correlation between the errors because all the $\mathbf{z_i}\mathbf{b}$ are independent (there is no repetition, dependence, in $\mathbf{z_i}$ if this is considered to be SampleID without repetition).

example of random errors occurring repeatedly versus not repeatedly

This is the basic consideration... Q:Do you repeat a measurement within the same class for which you may consider a random effect that occurs repeatedly? After this consideration you may wonder about additional things like whether your random effect is crossed vs nested

  • $\begingroup$ SampleID is a unique value across all the samples. For the wine case, 4 categories, each with 100 samples, there are 400 unique values of SampleID. In practice just an integer from 1...400. $\endgroup$ – sportscan Jun 29 '18 at 5:45
  • $\begingroup$ @sportscan Then it is like the right picture instead of the left. The 400 equations that you use to describe you statistical model have 400 unique error terms one in each line, there is no repetition of the same error term in multiple measurements. (in the left image you see each error terms, associated with the selected wine, occurring multiple times) $\endgroup$ – Sextus Empiricus Jun 29 '18 at 6:31
  • $\begingroup$ Yes, I think it is the situation in the image on the right. There would be 4 such images, one for full-bodied, and three others for the other three subjective categories (not mentioned in the OP, but we can imagine "fruity", "dry", and "hesitant") $\endgroup$ – sportscan Jun 30 '18 at 1:42
  • $\begingroup$ Thank you- though I do not yet think the suggestion of manova or LDA is appropriate. If there was no random effect, we would say that ANOVA is the obvious setup for difference in means between two or more groups defined by the categories. The only change from that is the random effect generated by sampling from a larger population (100 individuals sampled from the larger population of people with disease x). I do not believe that the random effect within each category would generate the "coupling" that would justify LDA. $\endgroup$ – sportscan Jun 30 '18 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.