3
$\begingroup$

I understand that there is a function in R called poly() that can generate orthogonal polynomials--useful for applying on input variables before running a predictive model.

My question is that what is the role of categorical variables when we generate polynomials? Are they to be excluded?

Update:

Dan, Thank you for your kind response. I'm not sure I understand it completely - let me explain the query in more detail. I'm trying to run logistic regression using glmnet on Titanic dataset.
Let us assume shortened set of columns:

    * class(factor with three levels 1, 2 ,3),
    * sex(factor: male, female),
    * Age (integer),
    *survived(factor & target variable 0 or 1).
The questions is it meaningful to create polynomial features based on these factors? e.g. class. If yes could you pls explain what it means?
I've seen examples with numeric input variables, where one can pass the entire input set to the poly() function and get polynomial features as output.
Your response is highly appreciated.

Improve this question
$\endgroup$
7
  • $\begingroup$ Dan, Thank you for your kind response. I'm not sure I got it completely - let me explain the query in more detail. I'm trying to run logistic regression on Titanic dataset. $\endgroup$
    – Seth
    Aug 18, 2012 at 19:27
  • $\begingroup$ I understand now. Polynomials are numeric by definition, so they don't make sense for binary factors like sex. You could dummy-code class as numeric, but with only 3 levels, linear is the highest-order polynomial that is worth using (N data points can be fit perfectly by a N-1 order polynomial, so using a quadratic on 3 levels would just be connecting them rather than fitting them). $\endgroup$
    – Dan M.
    Aug 19, 2012 at 10:52
  • $\begingroup$ Dan, Thanks a LOT...that explains it. In real life modeling, do people do this: (a) dummy code factors and use poly() or (b)only use poly on numeric variables & pass factors as such? $\endgroup$
    – Seth
    Aug 19, 2012 at 12:24
  • $\begingroup$ In my field (Psychology/Cognitive Neuroscience) option (b) is fairly common. People typically model continuous variables like time, age, etc. with polynomials and treat categorical variables like sex, diagnosis, etc. as discrete levels. Option (a) is also used, but before you go that route, you should explore regression with ordered categorical variables. I haven't used it, but I think it's a relatively standard approach. This might help. $\endgroup$
    – Dan M.
    Aug 19, 2012 at 20:37
  • $\begingroup$ Not sure how to thank for your kind time - but thank you! It has been very informative. $\endgroup$
    – Seth
    Aug 20, 2012 at 12:22

1 Answer 1

Reset to default
3
$\begingroup$

It's a little hard to answer without a specific example, but in general you can use orthogonal polynomials on the continuous variables and still include the categorical variables. Here's how it might work on some random data:

#create random data
dat <- data.frame(Y = rnorm(100), X = rep(1:50, 2), Cat = as.factor(rep(c("A", "B"), 50)))
#create second order orthogonal polynomial
x <- poly((unique(dat$X)), 2)
#insert it into original data frame
dat[,paste("ot", 1:2, sep="")] <- x[dat$X, 1:2]
#run regression
m <- lm(Y ~ (ot1 + ot2)*Cat, data=dat)
summary(m)
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Dan - I understand polynomial features for numerical values e.g. let input X matrix contain columns(numerical) x1, x2, x3. poly(X,degree=2) would return polynomials combinations of input variables. However I do not understand creating polynomial features out of factors. $\endgroup$
    – Seth
    Aug 18, 2012 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.