Skip to main content
20 votes
Accepted

Is Logistic Regression a classification or prediction model?

In the case of logistic regression, you obtain an equation but the output is a binomial classification. This is a common misconception. Explicitly, a logistic regression does no classification, ...
Dave's user avatar
  • 64.3k
17 votes

Area under the ROC curve when there is imbalance: is there a problem, and if not, why does this rumor exist?

This is actually a very simple issue. The area under the ROC curve (AUROC) equals the Wilcoxon-Mann-Whitney-Somers concordance probability, a $U$-statistic, i.e., take all possible pairs of an ...
Frank Harrell's user avatar
11 votes
Accepted

Using regression where the ultimate goal is classification

Welcome to the site. The first part of the first choice seems much better to me; you could use some form of count regression, probably negative binomial (the assumptions of Poisson regression are ...
Peter Flom's user avatar
  • 124k
11 votes

Area under the ROC curve when there is imbalance: is there a problem, and if not, why does this rumor exist?

This seems to me to be a misunderstanding of the criticism of AUROC in the (strongly) imbalanced case. Rephrasing the argument made by Saito and Rehmsmeier, it's not that AUROC is affected by class (...
Eike P.'s user avatar
  • 3,098
11 votes
Accepted

Overfitting in randomForest model in R, WHY?

First, note that your approach for classification is not optimal and that word-vector approaches are more suitable for this kind of task, as pointed out by @smci in the comment section (a summary of ...
J-J-J's user avatar
  • 4,811
10 votes

Is Logistic Regression a classification or prediction model?

Assuming we're all clear what logistic regression is in terms of technical implementation, one can still use it for many things. You highlight two options: Prediction/giving a probability for "1&...
Björn's user avatar
  • 33.3k
10 votes

Probability threshold in ROC curve analyses

Logistic regression is not strictly a classification method. It produces a linear predictor, a function of all the predictor values, that estimates the log-odds of a binary outcome, which then can be ...
EdM's user avatar
  • 94.5k
9 votes

Area under the ROC curve when there is imbalance: is there a problem, and if not, why does this rumor exist?

Whether an ROC curve depends on class imbalance or not will depend on which ROC curve you are referring to. More specifically, in this answer, we will consider two types of ROC curves: "...
mhdadk's user avatar
  • 5,100
8 votes
Accepted

How do you refer to data that's not part of train/test/validation?

I'm going to assume that you encountered some ambiguity relative to that, and that you want a solution to clarify it in the future. In the context of prediction, "new observations", "...
J-J-J's user avatar
  • 4,811
8 votes

Is ROC curve unique?

Maybe I don't understand your notation, but I feel you are mixing up true probabilities, model outputs (which might be probability estimates) and events. given an event E, model output $f(\theta)$ (...
seanv507's user avatar
  • 7,200
8 votes

Linear algebra properties of a confusion matrix (eigenvalues, eigenvectors, and determinants)

The eigenvalues would really only reveal how many classes (single classifier) or how many classifiers are correlated with one another (multiple classifiers). But if you look at the quasi-diagonalized ...
wjktrs's user avatar
  • 860
8 votes

What data are used to find the final threshold for a medical diagnostic test?

I have three thoughts. First, shouldn't you be sampling from a population that is similar to the one you will eventually be testing? This is unlikely to be a "normal" population. We ...
Peter Flom's user avatar
  • 124k
7 votes

For a confusion matrix, is there a name for FP / (FP + FN)?

I never heard the name for it and Wikipedia lists most of the named metrics like this, so my guess would be that it does not have a name.
Tim's user avatar
  • 140k
7 votes
Accepted

For a confusion matrix, is there a name for FP / (FP + FN)?

I would call this the proportion of the misclassifications/mistakes that are false positives. The denominator is the total number of misclassifications/mistakes. Of these mistakes, some are false ...
Dave's user avatar
  • 64.3k
7 votes
Accepted

Machine learning classification: best way to know if my variables are unable to distinguish between two classes

This is going to be hard. In this answer of mine, I give a situation where the two categories have dramatically different distributions, but only when the entire distribution in two dimensions is ...
Dave's user avatar
  • 64.3k
7 votes
Accepted

Probability in a classification problem

Yes, though to be a little more precise, in this case X% is the probability of being a dog (and 100-X% is the probability of being a cat) according to the model. Models can be very wrong and different ...
mkt's user avatar
  • 18.9k
7 votes
Accepted

Can a ML classifier's prediction be understood as a probability?

That would be desirable, but it is not guaranteed to make as much sense as we might like. First, you could make an argument that any predicted $p(\mathcal C_k|\mathbf x_i)\in[0,1]$ is a probability in ...
Dave's user avatar
  • 64.3k
7 votes

What data are used to find the final threshold for a medical diagnostic test?

If you oversample the minority class, your model parameters and predictions will be biased. This is usually not a good thing. However, if data collection is expensive, it may be an avenue worth ...
Stephan Kolassa's user avatar
6 votes

Classification error when estimating population size of rare phenomena

If you just want to estimate the prevalence of the phenomenon in the population, rather than identify the the particular instances of the phenomenon, then I would use e.g. logistic regression to make ...
Dikran Marsupial's user avatar
6 votes

Feature with multiple choice solutions in Logit

A question where respondents can pick several non-mutually exclusive options can simply be treated as a series of multiple binary questions, because this is what this kind of question actually is. In ...
J-J-J's user avatar
  • 4,811
6 votes
Accepted

How to interpret the results of a classifier when train/test method gives much better results than cross validated one?

What does these varying scores represent, particularly the low scores of cross validation? Together, they represent the fact that error estimtes based on a small number of tested cases are highly ...
cbeleites unhappy with SX's user avatar
6 votes
Accepted

Interpretation of a decision tree plot

The answer to your first question depends on what, exactly, you mean by "easier". The answer to your second question is "no". The fact that A was found on the first branch simply ...
Peter Flom's user avatar
  • 124k
5 votes

Is Logistic Regression a classification or prediction model?

The definitions you mention are imprecise and wrong. Others already commented on this, so I won't be repeating what was said just add my three cents. One can say that a prediction model is one that ...
Tim's user avatar
  • 140k
5 votes

Using regression where the ultimate goal is classification

“High risk” of what? You are predicting the number of failures, so if you aim to predict if there's a risk of failure, anything close to or higher than 1 failure is a risk. On another hand, if you ...
Tim's user avatar
  • 140k
5 votes

Can translation invariance be achieved by just a global pooling layer?

Intuitively, the problem I see is that by not being translation invariant the following layers after this Conv layer will receive different outputs and this will make their job harder during training ...
gunes's user avatar
  • 57.7k
5 votes

Classification error when estimating population size of rare phenomena

This topic has been covered to the extreme on this site. Briefly, when either class A or class B are extremely rare, classification is an inappropriate goal because as you have so rightly noted you ...
Frank Harrell's user avatar
5 votes
Accepted

Methods to derive cut-offs for continuous variables

Don't do this, as it particularly doesn't make sense for a random forest model. In addition to the many reasons that categorizing a continuous predictor is a bad idea, it undercuts a potential ...
EdM's user avatar
  • 94.5k
5 votes

Neural Network Classification - targetting class probability and not the class themselves

Consider the binary cross-entropy loss $L = -y \log f(x) + (1-y) \log (1-f(x))$ with binary labels $y \in \{0,1\}$ and our model produces predictions $f(x)$ from the features $x$. If we $n$ ...
Sycorax's user avatar
  • 92.3k
5 votes

How to deal with extremely small training data?

900 rows should be enough to at least test some model assumptions. If you consider this an extremely small data set, I would assume you come from a more AI-heavy background. As such, I would not ...
MrMidnight's user avatar
5 votes

Taking into account a non-symmetric loss function in a classification problem

I would recommend separating the modeling and the decision aspect: model probabilistically, assess these probabilistic classifications using proper scoring rules, and use thresholds when turning these ...
Stephan Kolassa's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible