problem with simple regression in R: one continuous response variable vs. two interacting categorical variables

Question

I have a dataset with a single continuous response variable, z, dependent on two categorical variables: x and y. x and y are applied individually, and together, and I interested in both the individual effects and any interaction effects.

I am hitting collinearity in my predictor variables, but I can't see it -see below. Is my encoding wrong, or my data too incomplete?

A toy-version of my data in R:

test_df <- data.frame(
  x = c(0,0,0,0, 1,1,1,1, 1,1,1,1),
  y = c(1,1,1,1, 0,0,0,0, 1,1,1,1),
  z = c(1.1,2.1,1.5,1.2,
        1.5,1.2,2,1.1,
        7,8,9,10)
) 
test_df$x <- as.factor(test_df$x)
test_df$y <- as.factor(test_df$y)
summary(lm(z ~ x * y, test_df))
alias(lm(z ~ x * y, test_df))

My problem is that I'm trying to recover the big jump in z when both x and y are '1'. In this case the lm gives this result:

Coefficients: (1 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)   
(Intercept)  -0.5750     0.7128  -0.807  0.44067   
x1            2.0250     0.5820   3.479  0.00695 **
y1            2.0500     0.5820   3.522  0.00649 **
x1:y1             NA         NA      NA       NA 

What am I doing wrong here? Variables x and y don't seem manifestly dependent. I've been playing with different amounts of toy data and different encoding schemes, to no avail. Thanks.

Answer 1

The problem here is that your interaction term is linearly dependent on the other three terms in your model: intercept, x, y. To see this use the model.matrix function which shows you all four columns of the matrix. Then notice that with the default encoding of factors the first three columns sum to 2 where the interaction column is 0 and to 3 where it is 1. This amounts to linear dependence. Your only solution would be to increase your sample until you find some observations which are zero for both x and y.

Answer 2

You have three conditions, not four. Your conditions are x only, y only, and x plus y. Thus, you can't test for an interaction in the way you want (as discussed in mdewey's answer). An alternative to adding the fourth condition of neither x or y is to approach this as a three group problem. Recall that with three categories of a factor (independent variable) you only have 2 degrees of freedom (hence your problem). You need to decide which effect coding answers your question. If you re-run the above model without the intercept, you will get a model that tests each condition (x, y, and x+y) against the grand mean. This might not be what you want. What are the specific contrasts you want to test? I recommend reading up on dummy and effect coding.

The effects for x and y in your current model are testing the difference between x versus x+y and y versus x+y given that x+y is being treated as the null or reference category. Other coding systems can test other effects but you can only have three coefficients in this model, including the intercept.