I am running a linear mixed effects model in which three of the regressor are inherently related. For sake of conceptual example: let's say I would like to see how the relative time employees arrive at work each day (early vs. late vs. exactly on time) influences their annual performance rating, over several years. Per year I have five pieces of data about each employee: the employee name, the performance rating, the percentage of days they arrived early, the percentage of days they arrived late, and the percentage of days they arrived exactly at 5pm.

The mixed model, accounting for the random effect of employee over multiple years/observations, would therefore be:

rating ~ PercentageEarly + PercentageLate + PercentageOnTime + (1|Employee)

Of course, these three fixed regressors are inherently related, as they together add up to 100 for each case observed. And indeed, when running this through lme4 package in R, the third regressor is dropped due to rank deficiency.

Obviously this would be a non-issue if only dealing with two such regressors. However, with three, how might the relative impact of each be considered, with separate coefficients? Alternatively, if only given a coefficient for the first two regressors (the current scenario), how might I interpret the relative impact of the third?

Apologies if this has been covered elsewhere; I wasn't sure of the exact terminology for describing such inherently related regressors and in turn may have missed a relevant posting.

  • $\begingroup$ It's not clear fom your post if you have data from a single year for these employees or from multiple years. If you only have one year's worth of data, you would be dealing with a linear regression model (not with a mixed effects linear regression model). $\endgroup$ – Isabella Ghement Jul 19 '19 at 1:20
  • 1
    $\begingroup$ Thanks, Isabella! Absolutely. To clarify, yes, there would be multiple observations (years) per employee in this example. I've updated the post to reflect this. $\endgroup$ – jjcii Jul 19 '19 at 15:36

for this model

 rating ~ PercentageEarly + PercentageLate + PercentageOnTime + (1|Employee)

PercentageOnTime was dropped out because PercentageEarly + PercentageLate + PercentageOnTime = 100. Then you should get estimate of the intercept ($\hat \beta_0$), slope for PercentageEarly ($\hat\beta_1$) and slope for PercentageLate ($\hat\beta_2$).

Then the mean of the rating is intercept ($\hat\beta_0$) if this person is 100% on time.

If not 100% ontime, the the mean of the rating is = $\hat\beta_0 + \hat\beta_1$PercentageEarly $+ \hat\beta_2$PercentageLate.

Let $X_1, X_2, X_3$ be percentageEarly, PercentageLate and PercentageOnTime with the restriction $X_1 + X_2 +X_3 =100$.

Let the fixed effect part is $$\beta_0 + \beta_1 X_1 +\beta_2 X_2 +\beta_3 X_3 =X\beta$$ Obviously, $X$ is less than full rank because $X_1 + X_2 +X_3 =100$, so we need to drop one column of $X$.

Beginning from drop $X_3$, we have $$\beta_0' + \beta_1' X_1 +\beta_2' X_2 =X_{-3}\beta_{-3}$$ If $X_2$ is dropped, we have

$$\beta_0'' + \beta_1'' X_1 +\beta_3'' X_3 = \beta_0'' + \beta_1'' X_1 +\beta_3'' (100-X_1-X_2) = X_{-3}\left(\array{1 & 0 & 100\\ 0 &1 &-1\\ 0 &0 &-1 } \right) \beta_{-2}$$ $$= X_{-3}A\beta_{-2}=X_{-2}\beta_{-2}$$ Then we have $$A\beta_{-2} = \beta_{-3}$$. Furthermore: $$\beta_0''=\beta_0'+100\beta_2'$$ $$\beta_1''=\beta_1'- \beta_2'$$ $$\beta_3''= - \beta_2'$$ Following this method, you can find the transformation of regression coefficients between the models generated by drop different items.

| cite | improve this answer | |
  • $\begingroup$ Thanks! To clarify, should "slope for PercentageOnTime (𝛽̂ 2)" actually be "slope for PercentageLate (𝛽̂ 2)", as PercentageOnTime was dropped? Additionally, I'm not quite certain what you mean by "Then on time effect is intercept (𝛽̂ 0)", etc. I do see how 𝛽̂ 0 + 𝛽̂ Late, when OnTime is omitted, is equal to 𝛽̂ 0 when Late is omitted instead. Is there a suggested means by which to report the individual effect of a given regressor in this case? Or must it be reported in terms of contrasts to an omitted baseline, similar to a categorical predictor (see @Parseltongue comment below)? $\endgroup$ – jjcii Jul 19 '19 at 15:33
  • $\begingroup$ I modified my answer. $\endgroup$ – user158565 Jul 19 '19 at 15:51
  • $\begingroup$ Thank you. The modifications further clarify the relation between intercepts and coefficients across models that drop alternate terms. To confirm, while this permits insight transformation of coefficients between models, any reporting should be of a given term in contrast to the omitted term, correct? $\endgroup$ – jjcii Jul 20 '19 at 15:27
  • $\begingroup$ Yes, by dropping different items, totally you can have 4 different models. But in fact they are equivalent, and you can covert the regression coefficients from one model you fit to others. $\endgroup$ – user158565 Jul 20 '19 at 16:05

One of the covariates will automatically be dropped due to perfect collinearity. As such, the coefficients on the other variables will be interpreted relative to the left-out variable (the 'reference group').

You can cycle through reference groups to get the effect of variables relative to whichever variable is left out. The way to do this is to just manually leave whatever variable out of the regression you want as the reference group.

If you need some specific hypothesis test take a look at the emmeans library for specifying contrasts


The covariates will be interpreted relative to the left out baseline (as with categorical variables), but in the simple case of your linear model this is equivalent to how user158565 is describing it. Because the effect of PercentageEarly will include the intercept + the coefficient on PercentageEarly, whereas PercentageOnTime is just the intercept, the coefficient on PercentageEarly is the additional amount (relative to PercentageOnTime that you get in terms of your outcome.

| cite | improve this answer | |
  • $\begingroup$ Excellent; thanks. To confirm: as they are collinear, there would be no individual effect per regressor in this case? Rather, similar to categorical variables, the relative effect would need to be reported in terms of a contrast (e.g., selecting one of the three as a baseline)? $\endgroup$ – jjcii Jul 19 '19 at 15:38
  • $\begingroup$ See comments. Me and user158565 are saying the same thing. $\endgroup$ – Parseltongue Jul 19 '19 at 15:44
  • $\begingroup$ Thank you. I understand the logic, and indeed, cycling through dropping out different terms, I do indeed see how the intercept in a model where X is omitted is always equivalent to the intercept of X + the coefficient of X in models where a different variable is omitted. In reporting, I would be required to select a baseline for contrasts and only report the relative effect of the other two regressors (e.g., "Compared to the percentage of days coming in late, the percentage of days coming in on time was had XX effect on annual ratings")? $\endgroup$ – jjcii Jul 20 '19 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.