1
$\begingroup$

I have a dataset with a single continuous response variable, z, dependent on two categorical variables: x and y. x and y are applied individually, and together, and I interested in both the individual effects and any interaction effects.

I am hitting collinearity in my predictor variables, but I can't see it -see below. Is my encoding wrong, or my data too incomplete?

A toy-version of my data in R:

test_df <- data.frame(
  x = c(0,0,0,0, 1,1,1,1, 1,1,1,1),
  y = c(1,1,1,1, 0,0,0,0, 1,1,1,1),
  z = c(1.1,2.1,1.5,1.2,
        1.5,1.2,2,1.1,
        7,8,9,10)
) 
test_df$x <- as.factor(test_df$x)
test_df$y <- as.factor(test_df$y)
summary(lm(z ~ x * y, test_df))
alias(lm(z ~ x * y, test_df))

My problem is that I'm trying to recover the big jump in z when both x and y are '1'. In this case the lm gives this result:

Coefficients: (1 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)   
(Intercept)  -0.5750     0.7128  -0.807  0.44067   
x1            2.0250     0.5820   3.479  0.00695 **
y1            2.0500     0.5820   3.522  0.00649 **
x1:y1             NA         NA      NA       NA 

What am I doing wrong here? Variables x and y don't seem manifestly dependent. I've been playing with different amounts of toy data and different encoding schemes, to no avail. Thanks.

$\endgroup$

2 Answers 2

4
$\begingroup$

The problem here is that your interaction term is linearly dependent on the other three terms in your model: intercept, x, y. To see this use the model.matrix function which shows you all four columns of the matrix. Then notice that with the default encoding of factors the first three columns sum to 2 where the interaction column is 0 and to 3 where it is 1. This amounts to linear dependence. Your only solution would be to increase your sample until you find some observations which are zero for both x and y.

$\endgroup$
3
$\begingroup$

You have three conditions, not four. Your conditions are x only, y only, and x plus y. Thus, you can't test for an interaction in the way you want (as discussed in mdewey's answer). An alternative to adding the fourth condition of neither x or y is to approach this as a three group problem. Recall that with three categories of a factor (independent variable) you only have 2 degrees of freedom (hence your problem). You need to decide which effect coding answers your question. If you re-run the above model without the intercept, you will get a model that tests each condition (x, y, and x+y) against the grand mean. This might not be what you want. What are the specific contrasts you want to test? I recommend reading up on dummy and effect coding.

The effects for x and y in your current model are testing the difference between x versus x+y and y versus x+y given that x+y is being treated as the null or reference category. Other coding systems can test other effects but you can only have three coefficients in this model, including the intercept.

$\endgroup$
1
  • $\begingroup$ Thank you. Both your answer and mdewey's are useful: mdewey's because it helps me understand the linear dependence, and yours because it gives me suggestions to think about the encoding for the contrast I'm trying to see - specifically the gain in applying the treatments x+y simultaneously, over the sum of effects applying x and y independently. $\endgroup$
    – Vivek Iyer
    Apr 28, 2017 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.