25
$\begingroup$

I am confused with the assumption of linearity to the logit for continuous predictor variables in logistic regression analysis. Do we need to check for the linear relationship while screening for potential predictors using univariable logistic regression analysis?

In my case, I am using the multiple logistic regression analysis to identify factors associated with nutritional status (dichotomous outcome) among the participants. The continuous variables including age, Charlson comorbidity score, Barthel Index score, hand grip strength, GDS score, BMI etc. My first step is to screen for significant variables using simple logistic regression. Do I need to check for the linearity assumption during simple logistic regression analyses for each continuous variables? Or should I just check for it in the final multiple logistic regression model?

Besides, for my understanding, we need to transform the non-linear continuous variable before enter it into the model. Can I categorize the non-linear continuous variable instead of transformation?

$\endgroup$
1
  • 3
    $\begingroup$ You should not categorize, better to try splines! $\endgroup$ Commented May 23, 2018 at 16:19

5 Answers 5

15
$\begingroup$

Logistic regression does NOT assume a linear relationship between the dependent and independent variables. It does assume a linear relationship between the log odds of the dependent variable and the independent variables (This is mainly an issue with continuous independent variables.) There is a test called the Box-Tidwell that you can use for this. The stata command is boxtid. I don't know the SPSS command, sorry.

This resource may be of help -- resource.

$\endgroup$
2
14
$\begingroup$

As I describe in detail in my book Regression Modeling Strategies (2nd edition available 2015-09-04, e-book available now), the process of attempting to transform variables before modeling is frought with problems, one of the most important being the distortion of type I error and confidence intervals. Categorization causes even more severe problems, especially lack of fit and arbitrariness.

Instead of thinking about this as a "check for lack of fit" problem, it is better to think of it as specifying a model that is very likely to fit. One way to do this is to allocate parameters to the parts of the model that are likely to be strong and for which linearity is not already known to be a reasonable assumption. In this process one examines the effective sample size (in your case the minimum of the number of events and number of non-events) and allows complexity to the extent that the data's information content allows (using e.g. the 15:1 events:parameter rule of thumb). By pre-specifying a flexible additive parametric model one will only be wrong where it matters by omitting important interactions. Interactions should be pre-specified, generally speaking.

You can check whether nonlinearity was needed in the model with a formal test (made easy with the R rms package) but removing such terms when insignificant creates the inferential distortions I outlined above.

More details may be found at course notes linked to from https://hbiostat.org/rms

$\endgroup$
5
  • $\begingroup$ Sorry for not mentioned it earlier but I am not familiar with R and was using SPSS for the analyses. From the solution provided, is it means that if I use the effective sample size (15: 1), I can include all important factors (from the review) without checking for their linearity? $\endgroup$ Commented Aug 30, 2015 at 12:41
  • $\begingroup$ From the univariable logistic regression analyses I had done in my case, BMI, calf circumference, mid-upper arm circumference are all making a significant contribution to the simple logistic regression model of nutritional status (p<0.05). But they turned out didn't met the linearity assumption when I check the assumption using Box-Tidwell approach (for each simple logistic model). So i am not sure should I proceed to multiple logistic regression analysis with these predictors or not. $\endgroup$ Commented Aug 30, 2015 at 12:47
  • 7
    $\begingroup$ It is invalid to build models on the basis of univariable analysis. You are using a variant to forward stepwise regression which is known to cause a host of problems. $\endgroup$ Commented Aug 30, 2015 at 13:12
  • $\begingroup$ @FrankHarrell I think your suggestions are good (+!) but I suggest a visual approach that may be useful if an initial fit is disguising the problem. Let me know if you think it is appropriate, as it is something I concocted from my own curiosity about this issue (and a seeming lack of visual aids I have seen online for this). $\endgroup$ Commented Feb 26 at 10:36
  • $\begingroup$ Yes the graphical approach is problematic. (1) Too subjective (2) creates model uncertainty, i.e., standard errors too low, confidence intervals too narrow, P-values too low (3) it’s better to investigate the consequences of not fitting well than lack of fit per se. $\endgroup$ Commented Feb 26 at 13:29
3
$\begingroup$

I think that we should plot continuous variables and check for linearity before using them in a regression model. If linearity seems like a reasonable assumption, I think this will probably still hold in the final multivariable regression model in most cases, and if not, I think this might primarily be caused by interaction effects that you can correct for.

Yes, categorizing non-linear continuous variables is one option. The problems with this are that categories may in most cases seem arbitrary, and small differences in cut-off scores between categories may lead to different results (especially regarding statistical significance), and, depending on the number of categories and the size of your data, you may lose much valuable information in the data.

An alternative approach is to use a generalized additive model which is a regression model that can be specified as a logistic regression, but in which you can include non-linear independent variables as "smoother functions". Technically, this is not very complicated in R, but I don't know about other software packages. These models will identify non-linear relationships to the dependent variables, but a drawback might be that you won't get neat and tidy numbers in your output to present, but rather a visual curve that is tested for statistical significance. So it depends how interested you are in quantifying the effect of the non-linear variable on the outcome variable.

Finally, you can use generalized additive models as described above to test the assumptions of linearity in your logistic regression model, at least if you use R.

Take a look at this book (a very different field from yours, and mine, but that doesn't matter at all): http://www.amazon.com/Effects-Extensions-Ecology-Statistics-Biology/dp/0387874577/ref=sr_1_1?ie=UTF8&qid=1440928328&sr=8-1&keywords=zuur+ecology

$\endgroup$
1
  • $\begingroup$ I am not familiar with R and was using SPSS for the analyses. Sorry for not mentioned it earlier. Can I use Box-Tidwell approach (by creating an interaction term between continuous variable and its own natural log and adding the interaction term to the model) to check for the linearity assumption? $\endgroup$ Commented Aug 30, 2015 at 12:35
2
$\begingroup$

Since I don't know your data I don't know if combining those three variables -- the basic variable, its natural log, and an interactive term -- will be a problem. However, I know that in the past when I have considered combining three terms I often lose conceptual track of what I am measuring. You need to have a good handle on what you are measuring or you'll have trouble explaining your findings. Hope that helps!

$\endgroup$
1
$\begingroup$

I find a lot of the suggestions here for determining nonlinearity in logistic regression, while good, are more indirect approximations of the issue rather than visualizing the problem like you would in OLS. One thing you can do is create a component plus residual (CR) plot by plotting the predictor against the component and the residuals, or:

$$ \hat{\beta_i}X_i ~\text{versus } ~X $$

With some programming, one can achieve this by fitting the model, plotting the predictor against the components + residuals, and layer on a LOESS line to see if the line is linear or nonlinear. Below I have a custom-made function in R which achieves this that I wrote some time ago (forgive me for the long code below, I am copy-pasting it here since I am in a rush):

#### Logistic GLM(M) Linearity Check
log_linear_check <- function(model,
                             data,
                             predictor,
                             random = F){ # defaults to GLM
  
  # Require tidyverse
  require(tidyverse)
  
  # Toggle correct residuals
  type.res <- ifelse(random == F, # if GLM 
                     "deviance", # deviance residuals
                     "pearson") # if GLMM, Pearson used
  
  # Get probabilities
  probabilities <- predict(model, type = "response")
  
  # Get logits
  logits <- log(probabilities/(1-probabilities))
  
  # Calculate residuals
  residuals <- residuals(model, type = type.res)
  
  # Create a dataframe
  df <- data.frame(predictor = data[[predictor]], 
                   residuals = residuals,
                   logits = logits)
  
  # Create component + residual plot
  ggplot(df, aes(x = predictor,
                 y = residuals + logits)) + # component plus residual
    geom_jitter(
      width=.02,
      height=.02,
      alpha=.4,
      size=1,
      color = "gray"
    )+
    geom_smooth(
      method = "loess",
      color = "darkgreen",
      formula = y ~ x
    ) +
    theme_classic() +
    ylab("Partial Residual") +
    xlab(predictor)+
    ggtitle("Logistic Regression Linearity Check")+
    geom_vline(aes(xintercept = min(predictor)),
               linetype = "dashed")+
    geom_vline(aes(xintercept = max(predictor)),
               linetype = "dashed")+
    scale_x_continuous(n.breaks = 10)+
    scale_y_continuous(n.breaks = 10)
}

To check if it works, I will fit a standard logistic regression to the wesdr data in the gamair package in R, which is normally used to showcase logistic GAMs, thus making it a nice candidate for the function. Here I fit the model with a typical GLM using the glm function:

#### Get Data and Inspect ####
library(gamair)
data("wesdr")
head(wesdr)

#### Fit Model ####
fit.linear <- glm(
  ret ~ dur, 
  data = wesdr,
  family = binomial
)

#### Check Linearity ####
log_linear_check(
  fit.linear,
  wesdr,
  "dur",
  random =F
)

You can see the CR plot shows substantial nonlinearity:

enter image description here

One can see when fitting a GAM instead that the default plots show very similar lines:

#### Load Libraries ####
library(mgcv)
library(gratia)

#### Fit GAM ####
fit.gam <- gam(
  ret ~ s(dur),
  data = wesdr,
  family = binomial,
  method = "REML"
)

#### Plot ####
draw(fit.gam)

enter image description here

$\endgroup$
2
  • $\begingroup$ Side comment: the R rms package` allows you to fit models and get such plots with 3-4 lines of code. $\endgroup$ Commented Feb 26 at 13:28
  • $\begingroup$ an example of this would be very helpful if you could provide one! $\endgroup$
    – pluke
    Commented Apr 4 at 9:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.