2
$\begingroup$

I am reading Elements of Statistical Learning and was looking at Chapters 4 and 5.

In chapter 4 logistic regression is performed on some 'South African Heart Disease Data'. The final model did not accurately model 2 parameters: systolic blood pressure, and obesity - the Z scores were found to be statistically insignificant (1.023 and -1.187). It was mentioned:

"Systolic blood pressure (sbp) is not significant! Nor is obesity, and its sign is negative. This confusion is a result of the correlation between the set of predictors. On their own, both sbp and obesity are significant, and with positive sign. However, in the presence of many other correlated variables, they are no longer needed (and can even get a negative sign)."

Model reduction was then performed to find a subset of variables but both of these predictors were still excluded.

In chapter 5 natural cubic splines are then used to fit the data. Plots of the splines for systolic blood pressure and obesity reveal they are inherently non-linear. Here are the plots:plots

The book then states:

" Both sbp and obesity are included in this model, while they were not in the linear model. The figure explains why, since their contributions are inherently nonlinear."

To me this is suggesting that logistic regression could not accurately model these non-linear predictors. However, just because they are non-linear in the predictors, surely provided their parameters are linear, logistic regression should still be able to model this?

So my question is : Is logistic regression not always able to accurately model non-linear predictors?

Any further clarification of what was going on in this example would also be appreciated.

$\endgroup$
6
  • 6
    $\begingroup$ I suspect you are misunderstanding what they did. logistic regression models the log odds as a linear function of the inputs. So you add non linear transformations (eg splines) of the inputs as extra inputs, and then logistic regression will model (logodds) as a nonlinear function of the original inputs. eg adding X^2 as an input as well as the original X $\endgroup$
    – seanv507
    May 30, 2020 at 13:11
  • 4
    $\begingroup$ You can also use splines of covariables as predictors in logistic regression, see here. $\endgroup$ May 30, 2020 at 17:54
  • $\begingroup$ thank you for both your comments. Yes, it seems i misunderstood the process, but it makes sense now! $\endgroup$
    – Sean
    May 30, 2020 at 18:54
  • $\begingroup$ @Sean Would you have any interest in posting a self-answer that summarizes what you learned in order to "close out" this question? Two additional advantages are that doing so forces you to think carefully about what you learned, which might reveal that there are gaps in your knowledge, and the community can respond to your post (comments, downvotes) if you do have a misconception. (If it turns out you got it wrong and get downvoted, you can always delete the post to remove the downvotes.) $\endgroup$
    – Dave
    Apr 2, 2023 at 17:59
  • 1
    $\begingroup$ @medium-dimensional I have updated the question text with the full name of the text. $\endgroup$
    – seanv507
    Dec 18, 2023 at 19:56

1 Answer 1

2
$\begingroup$

I suspect you are misunderstanding what they did.

Logistic regression models the log odds as a linear function of the inputs. But by adding nonlinear transformations (eg splines) of the inputs as extra inputs, logistic regression will model the log odds as a nonlinear function of the original inputs. eg adding $X^2$ as an input as well as the original X.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.