Dealing with rank deficiency when multiple regressors are inherently related / a non-binary ratio

Question

I am running a linear mixed effects model in which three of the regressor are inherently related. For sake of conceptual example: let's say I would like to see how the relative time employees arrive at work each day (early vs. late vs. exactly on time) influences their annual performance rating, over several years. Per year I have five pieces of data about each employee: the employee name, the performance rating, the percentage of days they arrived early, the percentage of days they arrived late, and the percentage of days they arrived exactly at 5pm.

The mixed model, accounting for the random effect of employee over multiple years/observations, would therefore be:

rating ~ PercentageEarly + PercentageLate + PercentageOnTime + (1|Employee)

Of course, these three fixed regressors are inherently related, as they together add up to 100 for each case observed. And indeed, when running this through lme4 package in R, the third regressor is dropped due to rank deficiency.

Obviously this would be a non-issue if only dealing with two such regressors. However, with three, how might the relative impact of each be considered, with separate coefficients? Alternatively, if only given a coefficient for the first two regressors (the current scenario), how might I interpret the relative impact of the third?

Apologies if this has been covered elsewhere; I wasn't sure of the exact terminology for describing such inherently related regressors and in turn may have missed a relevant posting.

Answer 1

for this model

 rating ~ PercentageEarly + PercentageLate + PercentageOnTime + (1|Employee)

PercentageOnTime was dropped out because PercentageEarly + PercentageLate + PercentageOnTime = 100. Then you should get estimate of the intercept ($\hat \beta_0$), slope for PercentageEarly ($\hat\beta_1$) and slope for PercentageLate ($\hat\beta_2$).

Then the mean of the rating is intercept ($\hat\beta_0$) if this person is 100% on time.

If not 100% ontime, the the mean of the rating is = $\hat\beta_0 + \hat\beta_1$PercentageEarly $+ \hat\beta_2$PercentageLate.

Let $X_1, X_2, X_3$ be percentageEarly, PercentageLate and PercentageOnTime with the restriction $X_1 + X_2 +X_3 =100$.

Let the fixed effect part is $$\beta_0 + \beta_1 X_1 +\beta_2 X_2 +\beta_3 X_3 =X\beta$$ Obviously, $X$ is less than full rank because $X_1 + X_2 +X_3 =100$, so we need to drop one column of $X$.

Beginning from drop $X_3$, we have $$\beta_0' + \beta_1' X_1 +\beta_2' X_2 =X_{-3}\beta_{-3}$$ If $X_2$ is dropped, we have

$$\beta_0'' + \beta_1'' X_1 +\beta_3'' X_3 = \beta_0'' + \beta_1'' X_1 +\beta_3'' (100-X_1-X_2) = X_{-3}\left(\array{1 & 0 & 100\\ 0 &1 &-1\\ 0 &0 &-1 } \right) \beta_{-2}$$ $$= X_{-3}A\beta_{-2}=X_{-2}\beta_{-2}$$ Then we have $$A\beta_{-2} = \beta_{-3}$$. Furthermore: $$\beta_0''=\beta_0'+100\beta_2'$$ $$\beta_1''=\beta_1'- \beta_2'$$ $$\beta_3''= - \beta_2'$$ Following this method, you can find the transformation of regression coefficients between the models generated by drop different items.

Answer 2

One of the covariates will automatically be dropped due to perfect collinearity. As such, the coefficients on the other variables will be interpreted relative to the left-out variable (the 'reference group').

You can cycle through reference groups to get the effect of variables relative to whichever variable is left out. The way to do this is to just manually leave whatever variable out of the regression you want as the reference group.

If you need some specific hypothesis test take a look at the emmeans library for specifying contrasts

@jjcii

The covariates will be interpreted relative to the left out baseline (as with categorical variables), but in the simple case of your linear model this is equivalent to how user158565 is describing it. Because the effect of PercentageEarly will include the intercept + the coefficient on PercentageEarly, whereas PercentageOnTime is just the intercept, the coefficient on PercentageEarly is the additional amount (relative to PercentageOnTime that you get in terms of your outcome.