1
$\begingroup$

I have noticed that Logistic Regression (https://en.wikipedia.org/wiki/Logistic_regression) is a model that used significantly for both Regression problems and Classification problems.

When used for Regression, the main purpose of Logistic Regression appears to be to estimate the effect of a predictor variable on the response variable. For example, here are some examples in which Logistic Regression is used for Regression problems:

When used for Classification, the main purpose of Logistic Regression appears to be to estimate the probability of the response variable assuming a certain value given an observed set of predictor variables. For example, here are some examples in which Logistic Regression is used for Classification problems:

Based on surveying such articles, I noticed the following patterns:

  • When Logistic Regression is being used for Regression problems, the performance of the Regression Model seems to be primarily measured using metrics that correspond to the overall "Goodness of Fit" and "Likelihood" of the model (e.g. in the Regression Articles, the Confusion Matrix is rarely reported in such cases)
  • When Logistic Regression is being used for Classification problems, the performance of the Regression Model seems to be primarily using metrics that correspond to the ability of the model to accurately classify individual subjects such as "AUC/ROC", "Confusion Matrix" and "F-Score".

The interesting thing being that regardless of whether you working on a Regression problem or a Classification problem - if you do decide to use Logistic Regression, in both cases you can calculate Classification metrics such as the Confusion Matrix. Based on these observations, I have the following question:

My Question: Suppose if I am using Logistic Regression in a regression problem (e.g. estimating the effect of predictors such as age on employment vs unemployment) and the model seems to be performing well (e.g. statistically significant model coefficients, statistically significant overall model fit, etc.). Even though I technically still able to calculate Classification metrics such as the Confusion Matrix, F-Score and AUC/ROC - am I still obliged to measure the ability of this Regression model to successfully classify individual observations based on metrics such as ROC/AUC? Or am I not obliged to this since I not working on a Classification problem?

I feel that it might be possible to encounter a situation/dataset in which the goal was to build a Logistic Regression model for a Regression problem - and the resulting model might have good performance metrics used in regression problems, but might have poor ROC/AUC values. In such a case, is this a good Logistic Regression model as it performs well for the regression problem as intended - or is it a questionable model as it is unable to perform classification at a satisfactory level?

$\endgroup$
7
  • 5
    $\begingroup$ It seems like you are interested in the difference between predictive and inferential modeling. Is that correct? // Either way, logistic regression is a method for estimating probabilities (or at least values between zero and one). $\endgroup$
    – Dave
    Apr 9, 2023 at 18:17
  • 1
    $\begingroup$ I must object to the downvoting of this question without the downvoter explaining why. $\endgroup$
    – Dave
    Apr 9, 2023 at 18:56
  • 1
    $\begingroup$ I didn't downvote but instead vote but instead vote to close this question. Surely whether a problem is a regression problem or a classification problem depends on what the goal of the study is. And the goal is not determined by the statistical analysis; it's the other way around. $\endgroup$
    – dipetkov
    Apr 10, 2023 at 16:12
  • 1
    $\begingroup$ @rep_ho Well to me it's not clear. There was a callout to explain why people might find something confusing about this question (under the assumption that clarity if subjective?) No instructions say that you are allowed to vote or comment only if you are interested in writing an answer. $\endgroup$
    – dipetkov
    Apr 10, 2023 at 19:30
  • 2
    $\begingroup$ I notice that you've, once again, added a bounty requesting answers from a reputable source without any elaboration. Moreover, the accepted answer already includes a citation. Are we to infer that the citation is not suitably reputable? What kind of information do you want to glean from a reference? On the other hand, if you do not really seek a reference, but instead want to attract more answers, it would make more sense to select "draw more attention" as the reason for the bounty. $\endgroup$
    – Sycorax
    Apr 12, 2023 at 3:50

1 Answer 1

5
+50
$\begingroup$

Some of the trouble with this question is that many of the "metrics typically associated to measure the performance of predictive modelling" are applied inappropriately, such as assessing the classification accuracy with no regard for how those classifications are created and if a software-default threshold (usually $1/2$) is appropriate.

That said, I have seen plenty of papers (published in the elite journals of their respective fields) where the ultimate goal is causal inference on the coefficients, yet the logistic regression models have standard metrics calculated like McFadden's $R^2$ or classification accuracy. For instance, Sundaram & Yermack (2007) report classification accuracy in their table 6 despite the main purpose of running those logistic regressions being the coefficient inference. (My take is that they made a mistake in doing so, because one of their models reports classification accuracy worse than would be achieved by predicting the majority class every time.) On the other hand, I recently saw another paper that had $R^2_{adj}<0$ all over the place. That their regression models were, arguably, doing worse than doing no modeling at all, helped me form a rationale for being skeptical of their results (there were other issues with the statistics). Thus, to explictly address your question, it can be helpful to give some sense of model performance, no matter how modest, even if prediction is not the main goal of the work. While a reviewer/professor/customer might not require it and oblige you to report it, I still believe it to be valuable information.

In such a case, it is typically acceptable to editors and referees to have rather modest performance in terms of the performance metrics. For instance, I have seen papers in top journals with adjusted $R^2$ around $0.05$, maybe even lower. Even that Sundaram & Yermack paper has rather pedestrian performance when accuracy scores are compared to naïvely predicting the majority category every time. (Most of their $R^2$-style scores, defined as I do in that link, are greater than zero (one is less than zero), but they do not scream out, "This model gets an $\text{A}$," the way that a classification accuracy of $97\%$ might.)

REFERENCE

Sundaram, Rangarajan K., and David L. Yermack. "Pay me later: Inside debt and its role in managerial compensation." The Journal of Finance 62.4 (2007): 1551-1588.

$\endgroup$
7
  • $\begingroup$ @ Dave: thank you so much for your answer! $\endgroup$
    – stats_noob
    Apr 9, 2023 at 19:56
  • 1
    $\begingroup$ @stats_noob What are you looking for in a bounty-worthy answer that is not addressed in my answer? The bounty message mentions wanting a reputable source. This answer gives a reference to an article from the top journal (“the” top journal, not “a” top journal) in its field, so what more would you want as a reputable source? $\endgroup$
    – Dave
    Apr 12, 2023 at 3:05
  • $\begingroup$ @ Dave: thank you for your comment! I am always interested in hearing the opinions of many of the experienced experts on this site. I learn so much from everyone! $\endgroup$
    – stats_noob
    Apr 12, 2023 at 3:06
  • 1
    $\begingroup$ @stats_noob It would help to clarify what you want out of a bounty-worthy answer. Perhaps such an answer will come from someone else, but it is hard to tell what you want when you post about wanting a reputable source after an answer is posted that refers to one of the top journals in the world. $\endgroup$
    – Dave
    Apr 12, 2023 at 3:12
  • $\begingroup$ @ Dave: sorry for the confusion, I will try to remember to be clearer next time! $\endgroup$
    – stats_noob
    Apr 12, 2023 at 3:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.