Treating predictors as numerical or categorical variable in regression

Question

I have a set of data that I am using regression analyses on. All of the columns are numeric (as far as I can see) a mix of integers and reals. However, two of the columns are being read from the CSV as factors, not numeric. I can't see any reason for this.

My first attempts at a regression were just to see how good each column was as a predictor (of time) on its own. The columns in question caused a bit of an issue. As factors they were excellent predictors alone (adjusted R${^2}$ of 0.45) however when converting them to numeric, they became poor predictors (adjusted R${^2}$ of 0.01).

I have three questions.

1. Why would these two columns be interpreted as factors rather than numeric;

2. Why would there be such an enormous change in the quality of the predictor

3. Can I justify using them (and other columns) as a factor?

Answer 1

There's not enough detail to be able to give more than a general answer:

(1) "Why would these two columns be interpreted as factors rather than numeric?" Non-numeric records in the column—leading/trailing spaces, commas as thousands separators, currency symbols, "Unknown", &c. Look & see: if it isn't obvious from a list of the factor levels, coercing to numeric will give missing values where records aren't being recognized as numbers.

(2) "Why would there be such an enormous change in the quality of the predictor?" Different models, as @Penguin_Knight pointed out: & perhaps different data-sets; as noted above coercing to numeric may well produce missing values, & the whole row may be excluded (by default) when fitting the model. If the former alone it may be an indication that a linear relationship between the predictor & response doesn't fit well.

(3) "Can I justify using them (and other columns) as a factor?" Possibly—depending on what the numbers represent & on what you want to do with the model. There are many ways to include any kind of variable in a model. E.g. suppose a predictor takes 10 distinct values in your data-set: it may be the case that a linear relationship between the predictor & response isn't a sensible assumption, yet you still want to be able to make predictions for values that aren't found in the data-set & don't want to spend 9 degrees of freedom modelling the relationship; so you represent the predictor with a low-order polynomial. Software, even R, can't do your thinking for you.

Answer 2

Here's an example using simulated data to explain why the factor model may reduce so much more error:

set.seed(50)
pred <- rep(1:5,each=10) # create our predictor variable that we will treat as numeric or as a factor
val <- c(sample(100,10),sample(200:500,10),sample(100,10),sample(-400:-390,10),sample(100,10)) # create our dependent variable.

As you can see, pred appears to predict val although there is no apparent linear relationship between them:

> head(cbind(pred,val))
     pred val
[1,]    1  71
[2,]    1  44
[3,]    1  20
[4,]    1  75
[5,]    1  50
[6,]    1   5

> cbind(pred,val)[11:16,]
     pred val
[1,]    2 317
[2,]    2 280
[3,]    2 391
[4,]    2 223
[5,]    2 282
[6,]    2 400

Now let's compared the $R^{2}$ values for linear models that treat pred as numeric and as factor:

> summary(lm(val~pred))$r.squared # numeric predictor
[1] 0.1764453
> summary(lm(val~as.factor(pred)))$r.squared # factor predictor
[1] 0.9710696

Wow! The our categorical predictor (as.factor(pred)) eats up so much more error! Why? It's a completely different model:

lm(val~pred) represents:

$Val_{i} = \beta_{0} +\beta_{1} Pred_{i} + \epsilon_{i}$

lm(val~as.factor(pred)) represents: $Val_{i} = \beta_{0} +\beta_{1} Pred_{pred = 2; i}+\beta_{2} Pred_{pred = 3; i} +\beta_{3} Pred_{pred = 4; i}+\beta_{4} Pred_{pred = 5; i}+ \epsilon_{i}$

Our categorical predictors do a much better job!