1
$\begingroup$

I would like to run logistic regression with categorical variable. My variable assumes the following categories x={red, green, blue}. So I have options to use two different encoding for them:

red = (1,0,0)
green = (0,1,0)
blue = (0,0,1)

or I can encode them as

red = (1,0)
green = (0,1)
blue = (0,0)

I am trying to figure out what effect different encoding will have on odds ratio? What approach is standard in the industry?

$\endgroup$
3
$\begingroup$

Mechanically, you can do either encodings, but be aware that if you do the former (i.e. dummies variables for red, green, and blue), then you cannot include a constant term in your regression because it will make your data matrix $X$ rank deficient!

On the other hand:

red = (1,0)
green = (0,1)
blue = (0,0)

will give the coefficients for red and green a natural interpretation as the log of the odds ratio relative to blue.

Another note is that your first encoding without a dummy is fundamentally the same regression as your second encoding with a dummy in the sense that a little linear algebra and math can figure out the results for one regression given the results of the other.

Detailed example about rank deficient data matrix:

Imagine you have 2 observations that are red, 2 observations that are green, and 2 that are blue. Every observation is either red, green, or blue. Your data matrix could be something like:

$$X = \begin{bmatrix} 1 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 1 \\ 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 1 \end{bmatrix}$$

The first column denotes a constant, the 2nd colum is an indicator for red, the 3rd column an indicator for green, and the 4th an indicator for blue. The matrix is rank deficint because the first column equals the sum of the other 3 columns! Full column rank would be 4, but this matrix is only rank 3.

$\endgroup$
  • $\begingroup$ Thank you for the explanation. Also great example for rand deficiency. I already heard somewhere that any n x (n+1) matrix is singular. $\endgroup$ – user1700890 Oct 10 '16 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.