1
$\begingroup$

How can I account for a nonlinear independent variable in a logistic regression?

For example, consider this data set:

b1  b2  b3  b4  b5
1   1   0   0   20
0   1   0   0   20
1   0   0   0   13
1   1   1   0   8
1   1   0   1   5
0   1   0   1   4
1   1   0   1   5
0   0   0   0   8
0   0   1   0   13
1   0   0   1   20
1   0   0   0   29
1   1   0   1   40
1   0   0   0   53
0   1   1   0   68

Suppose I gathered some data like the above (suppose I had 10,000 rows) and I want to use them to predict future sets of similar data with an equation generated by a logistic regression. I get the above data. My independent variables are b1 through b5. By looking at the data, I can clearly see that independent variables b1 through b4 are binary (maybe they are male/female, rent/own, etc).

However I see that b5 is clearly not binary. Upon graphing it and looking at the numbers I determine that the b5 variable has a bathtub shaped U curve with an equation of b5=(x-4)^2+4.

I understand that using the b5 variable as it is would screw up my logistic regression - since any interval independent variables must have a linear relationship with the dependent variable.

Yet, I must account for the variable b5 in my data set while doing a logistic regression for it to be accurate and predict well (right?).

Given the above data, how would I go about making a predictive model with a logistic regression? Any references online or in books would greatly be appreciated as I am having trouble finding a good explanation of the above.

Cheers =)

EDIT:

Sorry if my question is unclear.

Consider this: Suppose we had the following data on many people:

Y: Whether they had a stroke before the age of 50 or not 1. Whether they own a pet or not. 2. Whether they graduated college or not. 3. Whether they are male or female. 4. The number of hours they exercise every week.

We want to train a logistic regression model based on historical data to predict if someone in the future with the same data will be more apt or not to have a stroke before the age of 50.

The problem I am having is with the last variable. A continuous ratio variable like "number of hours exercised each week."

Exercising more each week may have diminishing returns on health and thus having a stroke. For example decreasing an increasing rate. Or maybe (totally made up) exercising a little bit is really good for you, exercising a moderate amount is really bad for you, and exercising a lot is really good for you again. We might have a U shaped curve like (x-4)^2+4 for example when plotting health to exercise.

I would image that having a curved, non-linear independent variable in a logistic regression like the one I described above would cause problems. Am I right? And if so, what kinds of things can you do to still include the variable in the logistic regression. Perhaps a transformation? Is that all?

$\endgroup$
7
  • 2
    $\begingroup$ y=(x-4)^2+4 implies your outcome is not binary valued. $\endgroup$ – AdamO Dec 10 '13 at 21:40
  • 2
    $\begingroup$ What are "y" and "x"? They do not seem to appear anywhere in the dataset. $\endgroup$ – whuber Dec 10 '13 at 21:42
  • 5
    $\begingroup$ "Nonlinear" pertains to a relationship between 2 (or more) variables; saying that a single variable is nonlinear is meaningless. $\endgroup$ – gung - Reinstate Monica Dec 10 '13 at 21:43
  • 3
    $\begingroup$ If $Y$ is binary and that is your whole dataset, you do not have a sufficient sample size to even be able to estimate the intercept, must less the regression effects. $\endgroup$ – Frank Harrell Dec 10 '13 at 22:03
  • 2
    $\begingroup$ I have great difficulty understanding how graphing b5 could show "a bathtub shaped U curve". What plot is this? You could have a bathtub shaped relationship between 2 different variables, but that is a property of the relationship b/t them, not a property of b5. Also, what is "x", there is still no x listed. $\endgroup$ – gung - Reinstate Monica Dec 11 '13 at 15:27
9
$\begingroup$

As written, your question can't work, since y is a 0-1 variable and you're doing logistic regression.

If you mean that the linear predictor had a nonlinear relationship with one of the independent variables, that is, $\eta = a + bf(x)$, say, for some nonlinear $f$ (with all other variables held constant), then you can write $x^* = f(x)$ and put $x^*$ in your logistic regression as an independent variable. [In a logistic regression, $\eta = \text{logit}(P[Y=1])$]

This is quite commonly done in linear models and generalized linear models; there's a linear relationship, but it's with a transformed independent variable. Under the usual assumptions you need for a GLM, the transformed variable works perfectly well as a predictor.

Note that if $f$ is known and that coefficient, $b$ is known, you don't put $x^*$ in as a predictor, because $x^{**} = bf(x)$ is then an alternative predictor with coefficient 1; those come in as offsets (e.g. specified in R by using the offset argument). (In ordinary regression you could let $y^* = y-x^{**}$ instead, for the same effect.)

I will assume the coefficient of $f$ is unknown (though you specified it to be 1).

In your particular case $x^* = f(x) = (x-4)^2$. If you were unsure about the "4" there (e.g. if it's just a rough guess or something, rather than a value that's definitely known), then you could instead use two new variables, $x^*_l = x-4$ and $x^*_q = (x-4)^2$ both as predictors, which will capture a general quadratic relationship (with the additional benefit that if the '4' is nearly right, the estimates be nearly uncorrelated with each other and with the intercept.

$\endgroup$
11
  • $\begingroup$ Thank you for your help. Your response is a little over my head, but yes, you are correct about what my question is. I am just not sure how to handle continuous nonlinear independent variables in a logistic regression. It feels like a mistake to me to use a continuous independent variable that is nonlinear in the logistic regression, so I am looking for a solution to that. A transformation like you said sounds like what I need to do. Just fuzzy on how to actually do it correctly with a logistic regression. $\endgroup$ – Micro Dec 11 '13 at 17:40
  • 1
    $\begingroup$ cyborglizard - several people have asked you to clarify this point, and as far as I can see, you still haven't done so in a way any of us can clearly understand (I took a guess for my answer). When you say "continuous nonlinear independent variables" what is nonlinear with respect to what else? A thing isn't nonlinear by itself. Are you referring to the index order (the order the observations are in) as your 'x'? Why would that be relevant? What variable is represented by that order? Are these observations over time or along a spatial dimension or something? What is b5, exactly? $\endgroup$ – Glen_b Dec 11 '13 at 21:49
  • $\begingroup$ I hope the example I provided above explains those questions. $\endgroup$ – Micro Dec 12 '13 at 15:00
  • $\begingroup$ cyborglizard, there's much you still need to clarify. You say the relationship of health to exercise may not be linear. The first difficulty is that 'health' is not a variable in your regression - you have introduced a new variable and not related it to the actual Y in your problem. The second difficulty is that the relationship you propose in your new discussion is not the relationship you have inserted into your data. In your example data, b5 is quadratically related to some unstated variable $x$. In your new discussion, b5 is $x$! Such confusions have to be cleared up. ... (ctd) $\endgroup$ – Glen_b Dec 12 '13 at 20:03
  • $\begingroup$ (ctd) ... However, if 'health' corresponds to the linear predictor in the GLM, (so with other variables held constant $P(Y=1) = \exp(\alpha_h+\beta h)/(1+\exp(\alpha_h+\beta h))$), where Y is the probability of stroke, $h$ is 'health' and $\alpha_h$ is a constant term depending on what values the other variables are held constant at), and health is itself linearly related to everything but exercise, ... then an appropriate approach is the one covered in my answer. $\endgroup$ – Glen_b Dec 12 '13 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.