Say I am building a regression model with Gauss-Markov assumptions for predicting the quality of a movie - either a linear regression for continuous rating or logistic regression for if a customer recommends it. We are only considering movies that Al Pacino, Robert de Niro or Joe Pesci ever starred in. In the model I have the following three indicator functions:

$${X_1}=1\{Al\ Pacino\ is\ in\ it\}$$ $${X_2}=1\{De\ Niro\ is\ in\ it\}$$ $${X_3}=1\{Joe\ Pesci\ is\ in\ it\}$$

Now in addition, I am interested in the total number of actor appearances among the above three. Therefore I add a categorical variables representing the sum of them:

$$X'_4=X_1 + X_2+X_3$$

When I implement this system in R, the whole system (ignoring any constant terms) can only have four degrees of freedom. Therefore, I arbitrarily let:

$$X_4=1\{X_1 + X_2+X_3==1\}$$

I am having difficulty of understanding why exactly the resulting degree of freedom would be five instead of six since, say, $1\{X_1 + X_2+X_3==3\}$ cannot be expressed as a linear combination of $X_1, X_2, X_3$. In addition, in my formulation with $X_1, X_2, X_3, X_4$, what is the reference level?


    my_df <- tibble(
  x1 = (c(1,0,0,1,0,1,1)), 
  x2 = (c(0,1,0,1,1,0,1)), 
  x3 = (c(0,0,1,0,1,1,1)),
  z = x1 + x2 + x3,
  y = c(1,0,1,1,1,0,1)

summary (aov (my_df$y ~ factor(my_df$x1) + factor(my_df$x2) +factor(my_df$x3) + factor(my_df$z) ))

Edit: Thanks to Alex's comment, I realised that I had one constraint that $X_1 + X_2 + X_3 >= 1$, i.e. only considering movies that any one of the three actors starred in. The edit has been added in the original question.

I was able to figure out formally but still not intuitively I think. Formally I will show that the system described above can have only 5 degrees of freedom (df). Suppose we have 6 df instead. Without loss of generality, let: $${X_1}=\{1,0,0,1,1,0,1\}$$ $${X_2}=\{0,1,0,1,0,1,1\}$$ $${X_3}=\{0,0,1,0,1,1,1\}$$ $${Z_1}=\{1,1,1,0,0,0,0\}$$ $${Z_2}=\{0,0,0,1,1,1,0\}$$ $${C_0}=\{1,1,1,1,1,1,1\}$$ Notice $Z_1 = (X_1+X_2+X_3 == 1)$, $Z_2 = (X_1+X_2+X_3 == 2)$, and $C_0$ is a constant. Further, we can express $$Z_2 = 3C_0 - (X_1+X_2+X_3) -2Z_1$$ The system is singular. Therefore it cannot have six df.

In spite of that, I still have trouble intuitively understand what the reference level would be if, say, I am using $X_1, X_2, X_3, Z_1, C_0$.


You don't have enough distinct levels of z in your dataframe (z only takes the values 1,2 and 3). If you add a new row with x1=x2=x3=0 (hence z=0) you get 5 df.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.