Skip to main content
1 of 6
stats_noob
  • 1
  • 6
  • 39
  • 124

Measuring the Performance of Logistic Regression: Regression vs Classification

I have noticed that Logistic Regression (https://en.wikipedia.org/wiki/Logistic_regression) is a model that used significantly for both Regression problems and Classification problems.

When used for Regression, the main purpose of Logistic Regression appears to be to estimate the effect of a predictor variable on the response variable. For example, here are some examples in which Logistic Regression is used for Regression problems:

When used for Classification, the main purpose of Logistic Regression appears to be to estimate the relationship between a set of predictor variables and the response variable - specifically, the probability of the response variable assuming a certain value given an observed set of predictor variables. For example, here are some examples in which Logistic Regression is used for Classification problems:

Based on surveying such articles, I noticed the following patterns:

  • When Logistic Regression is being used for Regression problems, the performance of the Regression Model seems to be primarily measured using metrics that correspond to the overall "Goodness of Fit" and "Likelihood" of the model
  • When Logistic Regression is being used for Classification problems, the performance of the Regression Model seems to be primarily using metrics that correspond to the ability of the model to accurately classify individual subjects such as "AUC/ROC" and "F-Score".

Based on these observations, I have the following question:

My Question: Suppose if I am using Logistic Regression in a regression problem (e.g. estimating the effect of predictors such as age on income) and the model seems to be performing well (e.g. statistically significant model coefficients, statistically significant overall model fit, etc.), am I obliged to also measure the ability of the model to successfully classify individual observations based on metrics such as ROC/AUC? Or am I not obliged to this since I not working on a classification problem?

I feel that it might be possible to encounter a situation/dataset in which the goal was to build a Logistic Regression model for a Regression problem - and the resulting model might have good performance metrics used in regression problems, but might have poor ROC/AUC values. In such a case, is this a good Logistic Regression model as it performs well for the regression problem as intended - or is it a questionable model as it is unable to perform classification at a satisfactory level?

Thanks!

stats_noob
  • 1
  • 6
  • 39
  • 124