5
$\begingroup$

I just need a direction on which regression algorithm (preferably glm or similar) algorithm to use when the predictor variables are a mix of numerical and categorical variables. The output is numerical for the time being but in future, I need to extend this for categorial output also.

My input columns are in the the format given below: 

preNum1 predNum2 ......predCat1 predCat2 ....ResponseVar

Edit: I am trying to predict the amount of 'Bilirubin' (here, the response variable), depending on patients' data. We have patients's lab results such as glucose, blood pressure, etc etc as numerical predictor values and the patients' disease history (diabetes,and other diseases) as categorical predictors. I converted all the categorical variables (they are diseases such as diabetes, high BP, etc) into 0 and 1 (0 representing that it is not present, 1 representing it to be present.) After this I applied GLM on this. I am wondering if my approach is correct or not?

Improve this question
$\endgroup$
5
  • $\begingroup$ Could you be more specific about your model? Are these categorical variables (diseases) predictors or outcomes? What software did you use to calculate the GLMs? What was the input command? $\endgroup$
    – COOLSerdash
    Commented May 13, 2013 at 14:21
  • $\begingroup$ @COOLSerdash, Thanks for reply. please see the edited OP $\endgroup$
    – user1140126
    Commented May 13, 2013 at 14:30
  • 1
    $\begingroup$ Your approach seems fine. It's a simple linear model and can be fitted with ordinary least squares. Categorical responses (outcome/dependent variables) create all sorts of problem addressed with other methods but categorical predictors do not require anything else as such. $\endgroup$
    – Gala
    Commented May 13, 2013 at 14:51
  • $\begingroup$ @GaëlLaurans, I was more concerned with converting the categorical predictors into zeros and ones. Thanks for your insight $\endgroup$
    – user1140126
    Commented May 13, 2013 at 15:19
  • $\begingroup$ @user1140126: Even if you are unaware of it: internally, the software converts categorical variables automatically into dummy variables (0,1): in R with the command as.factor and in Stata with the i-Prefix. $\endgroup$
    – COOLSerdash
    Commented May 13, 2013 at 15:37

3 Answers 3

Reset to default
10
$\begingroup$

When your dependent variable is binary ($1$ vs. $0$, "dead" vs. "alive"), the you might use logistic regression which is a glm with a binomial error distribution and a logit link function. When your dependent variable is ordinal (e.g. "bad"> "good" > "best"), you can use ordinal logistic regression. For a nominal (e.g. transportation: "walk", "car", "bicycle") dependent variable, you can use multinomial logistic regression.

EDIT

Your approach to convert the disease status into a 0,1-variable seems correct. If your outcome is continuous, you could use a GLM with a gaussian error distribution and an identity link function which is equivalent to a simple multiple regression model (OLS).

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Bilirubin I would expect to be strongly skewed, so other error families and/or links seem indicated. $\endgroup$
    – Nick Cox
    Commented May 13, 2013 at 14:44
  • $\begingroup$ @Nick Cox: Good point. As usual, a check of the residuals (among other things) is clearly warranted. A priori, the dependent variable doesn't have to be normal for a regression with gaussian errors to be valid, but the residuals have to be. $\endgroup$
    – COOLSerdash
    Commented May 13, 2013 at 14:51
  • 2
    $\begingroup$ We agree on essentials, but the idea that you must have normal residuals is oversold. Even if the errors really are exactly normal, the residuals won't be. If the errors are (e.g.) uniform, the usual recipes will perform rather well; it's just that e.g. P-values won't be exactly trustworthy, but they never are, any way. $\endgroup$
    – Nick Cox
    Commented May 13, 2013 at 15:20
4
$\begingroup$

The algorithm used by any implementation of generalised linear models is immaterial here -- or at least if there are grounds to choose one algorithm rather than another they don't hinge on any detail you mention.

Typically you present categorical predictors to GLM commands or functions as indicator variables (dummy variables, in a common vulgar terminology).

Quite how you do this depends a little on the software, but at the most complicated you just need to calculate your indicators in advance.

If you have a more specific question, please ask it.

Improve this answer
$\endgroup$
1
$\begingroup$

glm requires that all variables be numeric. Typically, you convert categorical variables to numeric variables using dummy variables.

Improve this answer
$\endgroup$
3
  • 5
    $\begingroup$ Why call a model an algorithm? Blurs a key distinction! $\endgroup$
    – Nick Cox
    Commented May 13, 2013 at 14:08
  • $\begingroup$ @NickCox I always think of a specific formula as a model (e.g. y = mx+b), that can be evaluated using different approaches (algorithms). Is this incorrect? $\endgroup$
    – Abe
    Commented May 13, 2013 at 14:37
  • 2
    $\begingroup$ That's fine by me, and far more importantly a standard distinction. (I wouldn't choose your symbols m and b, but that's a completely separate point.) It follows that model and algorithm are not identical. The model is a statistical definition, while the algorithm is a computational recipe. $\endgroup$
    – Nick Cox
    Commented May 13, 2013 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.