2
$\begingroup$

I have a set of data that I am using regression analyses on. All of the columns are numeric (as far as I can see) a mix of integers and reals. However, two of the columns are being read from the CSV as factors, not numeric. I can't see any reason for this.

My first attempts at a regression were just to see how good each column was as a predictor (of time) on its own. The columns in question caused a bit of an issue. As factors they were excellent predictors alone (adjusted R${^2}$ of 0.45) however when converting them to numeric, they became poor predictors (adjusted R${^2}$ of 0.01).

I have three questions.

  1. Why would these two columns be interpreted as factors rather than numeric;

  2. Why would there be such an enormous change in the quality of the predictor

  3. Can I justify using them (and other columns) as a factor?

$\endgroup$
  • 5
    $\begingroup$ Because when modeled as factor, you were fitting an ANOVA to test if the means between all the levels inside that factor variable are equal to zero. When modeled as numeric, you were fitting a linear regression testing if the slope between the outcome and the numeric is zero. They are two different models. $\endgroup$ – Penguin_Knight Oct 20 '14 at 3:27
  • 1
    $\begingroup$ I'm not sure I completely understand, but from what I could gather the cause may be because each record had a fairly distinct (not quite unique) factor, as such the regression stated that most of the time, that one semi-unique factor predicted time perfectly. $\endgroup$ – Zack Newsham Oct 20 '14 at 4:52
  • $\begingroup$ Since you are using R, it will be useful if you can post outputs of commands 'str(mydata)' and 'head(mydata)' here. $\endgroup$ – rnso Oct 20 '14 at 14:58
3
$\begingroup$

There's not enough detail to be able to give more than a general answer:

(1) "Why would these two columns be interpreted as factors rather than numeric?" Non-numeric records in the column—leading/trailing spaces, commas as thousands separators, currency symbols, "Unknown", &c. Look & see: if it isn't obvious from a list of the factor levels, coercing to numeric will give missing values where records aren't being recognized as numbers.

(2) "Why would there be such an enormous change in the quality of the predictor?" Different models, as @Penguin_Knight pointed out: & perhaps different data-sets; as noted above coercing to numeric may well produce missing values, & the whole row may be excluded (by default) when fitting the model. If the former alone it may be an indication that a linear relationship between the predictor & response doesn't fit well.

(3) "Can I justify using them (and other columns) as a factor?" Possibly—depending on what the numbers represent & on what you want to do with the model. There are many ways to include any kind of variable in a model. E.g. suppose a predictor takes 10 distinct values in your data-set: it may be the case that a linear relationship between the predictor & response isn't a sensible assumption, yet you still want to be able to make predictions for values that aren't found in the data-set & don't want to spend 9 degrees of freedom modelling the relationship; so you represent the predictor with a low-order polynomial. Software, even R, can't do your thinking for you.

$\endgroup$
1
$\begingroup$

Here's an example using simulated data to explain why the factor model may reduce so much more error:

set.seed(50)
pred <- rep(1:5,each=10) # create our predictor variable that we will treat as numeric or as a factor
val <- c(sample(100,10),sample(200:500,10),sample(100,10),sample(-400:-390,10),sample(100,10)) # create our dependent variable.

As you can see, pred appears to predict val although there is no apparent linear relationship between them:

> head(cbind(pred,val))
     pred val
[1,]    1  71
[2,]    1  44
[3,]    1  20
[4,]    1  75
[5,]    1  50
[6,]    1   5

> cbind(pred,val)[11:16,]
     pred val
[1,]    2 317
[2,]    2 280
[3,]    2 391
[4,]    2 223
[5,]    2 282
[6,]    2 400

Now let's compared the $R^{2}$ values for linear models that treat pred as numeric and as factor:

> summary(lm(val~pred))$r.squared # numeric predictor
[1] 0.1764453
> summary(lm(val~as.factor(pred)))$r.squared # factor predictor
[1] 0.9710696

Wow! The our categorical predictor (as.factor(pred)) eats up so much more error! Why? It's a completely different model:

lm(val~pred) represents:

$Val_{i} = \beta_{0} +\beta_{1} Pred_{i} + \epsilon_{i}$

lm(val~as.factor(pred)) represents: $Val_{i} = \beta_{0} +\beta_{1} Pred_{pred = 2; i}+\beta_{2} Pred_{pred = 3; i} +\beta_{3} Pred_{pred = 4; i}+\beta_{4} Pred_{pred = 5; i}+ \epsilon_{i}$

Our categorical predictors do a much better job!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.