26
$\begingroup$

I just realized I have always worked regression problem where the independent variables were always numerical. Can I use linear regression in the case where all independent variables are categorical?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
30
$\begingroup$

Just some semantics and to be clear:

  • dependent variable == outcome == "$y$" in regression formulas such as $y = β_0 + β_1x_1 + β_2x_2 + ... + β_kx_k$
  • independent variable == predictor == one of "$x_k$" in regression formulas such as $y = β_0 + β_1x_1 + β_2x_2 + ... + β_kx_k$

So in most situations the type of regression is dependend on the type of dependent, outcome or "$y$" variable. For example, linear regression is used when the dependent variable is continuous, logistic regression when the dependent is categorical with 2 categories, and multinomi(n)al regression when the dependent is categorical with more than 2 categories. The predictors can be anything (nominal or ordinal categorical, or continuous, or a mix).

(The remark below might be redundant for you, but I add it anyway)

However, do note that most software requires you to recode categorical predictors to a binary numeric system. This just means coding sex to 0 for females and 1 for males or vice versa. For categorical variables with more than 2 levels you'll need to recode these into $L-1$ dummy variables where $L$ is the number of levels and these dummies contain a 0 or 1 when they are in the corresponding category. This way each individual (sample) should be represented by having a 1 for the dummy variable he/she is part of and a 0 for the others, or a 0 for all dummies when he/she is part of the reference group.

Improve this answer
$\endgroup$
8
  • 1
    $\begingroup$ thanks. as I write in the question title, the dependent variable is continuous. So I take your answer as "you can use linear regression, provided you do dummy encoding". Please correct me if I am wrong. $\endgroup$
    – famargar
    Commented Mar 13, 2017 at 9:29
  • $\begingroup$ yes that is what I was saying. $\endgroup$
    – IWS
    Commented Mar 13, 2017 at 9:37
  • 2
    $\begingroup$ I see you have edited the question to add a second question, and posted a similar quesiton here: stats.stackexchange.com/questions/267137/… . Additionally, I'd ask you what you mean by smoothing your predictions, or what you mean by predicting discrete values. AFAIK a linear regression will give you the mean value of the continuous dependent based on your predictor variables (through the regression formula). Please elaborate $\endgroup$
    – IWS
    Commented Mar 13, 2017 at 10:08
  • 1
    $\begingroup$ I deleted the second question as you fully answered the original one. To answer your question, if I feed $n$ new "events" ($x_i$) to the model, I would get $n$ different $y$ values that would all take one of four regressed values. I guess I am saying that if the categorical variables were actually ordinal, I would like to introduce some (logit?) smoothing between values. $\endgroup$
    – famargar
    Commented Mar 13, 2017 at 12:44
  • 1
    $\begingroup$ In the case of an ordinal variable one can always chose to assume it is "continuous enough" to use it as if it were a continuous predictor (by simply not using dummies, but entering the variable as a numerical version). However, if you do this and you have only few levels, you are fitting a straight line (thus assuming linearity) through only a few points (so note that the amount of levels is important here). A Likert scale is a good example of a variable used this way, which regrettably creates problems on various occasions. $\endgroup$
    – IWS
    Commented Mar 13, 2017 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.