New answers tagged classification
0
votes
Gaussian RBF vs KNN explanation
One thing has to be said at the beginning RBF is not classification algorithm. It is used in SVM as a kernel, because it has properties of the kernel. SVM doeas all the work with finding hyperplane. I ...
- 1
3
votes
Accepted
Selecting best classification probability threshold with ROC/AUC doesn't necessarily improve F1 score
You have two metrics, the F1-score on the one hand and Youden's J on the other hand. Those two metrics measure different aspects of the model which are not functions of each other, so you cannot ...
- 8,694
0
votes
How is the direction of intercept determined?
For linear regression, the intercept can be positive or negative (or 0), and simply represents the value of the mean of Y when the hyperplane (your linear relationship) passes through the point where ...
- 51
0
votes
Take into account uncertainty in ground-truth in a classification
It sounds like the uncertainty in your ground truth stems primarily from uncertainty about the true duration and intensity of the phenomena's presence: when the sensors show positive, it's not clear ...
- 934
1
vote
Was Amazon's AI tool, more than human recruiters, biased against women?
One additional thought, not fully explored: Maybe we should think whether overfitting can amplify existing bias.
In a linear model (where we understand better what happens), we observe overfitting ...
0
votes
Could we explain the disadvantage of imbalanced data mathematically?
Not an explanation but an example illustrating that strong imbalance forces dismissal of the underrepresented class.
Let $y = y_1, ..., y_N$ be a sample where $y_k = 0$ except for $y_1 = 1$. Also, ...
- 1,369
1
vote
F1 score for change point detection
The issue in changepoint detection is that there is no single measure that assesses all the important properties for changepoint detection. At a minimum you want a measure that assesses closeness to: ...
- 1,167
5
votes
Could we explain the disadvantage of imbalanced data mathematically?
There is indeed a bias with logistic regression and maximum likelihood estimation when the classes are not equal. Below is a demonstration by coding a simulation (example in literature here)
...
- 51.8k
0
votes
Best way to impute missing values in a binary variable
You would be better off using something like multiple imputation. There is a package in R called mice for this purpose. The imputation will essentially fill in the ...
7
votes
Could we explain the disadvantage of imbalanced data mathematically?
In general statisticians are not worried about bias in imbalanced data (not a problem per se), since
they use probabilistic classifiers like logistic regression
the bias (in small samples) of ...
- 4,548
10
votes
Could we explain the disadvantage of imbalanced data mathematically?
Not a formal proof, but an intuition: Unbalanced data is not a problem per se, the problem is that you don't have many samples to represent the minority class. Imagine a trivial model, where you would ...
Tim♦
- 121k
3
votes
Could we explain the disadvantage of imbalanced data mathematically?
Not all strongly unbalanced data would result in logistic regression underestimating the probability of the minority in a relevant way. If the data is perfectly separated (and logistic regression is ...
- 8,694
0
votes
Generalized classification accuracy metric that coincides with accuracy metric when classes are balanced
Yes, there is such a metric and it's called balanced accuracy. It's the arithmetic mean of true positive and negative rates (TPR and TNR(. Here is some intuition on it. The balancing logic explained ...
- 53.2k
1
vote
Accepted
What happens if we change the threshold probability value for classifying into different class?
In addition to @Dave's pointer in his comment, I'd like to add simply that a threshold is a hyperparameter. You pick the one that gives you the best compromise between false positives and false ...
- 2,124
0
votes
How to compute precision/recall for multiclass-multilabel classification?
In case if you want to see the results directly:
from sklearn.metrics import classification_report, confusion_matrix
classification_report(y_test, y_pred)
This ...
5
votes
Accepted
Binary classification cross validation ROC score - only consider higher confidence class probabilities
When I do this, I lose about 85% of the test samples, but the resulting ROC score of the high confidence test set is boosted to 0.87, which makes it useful for downstream analysis.
It sounds to me ...
- 101k
5
votes
Binary classification cross validation ROC score - only consider higher confidence class probabilities
The ROC curve and corresponding AUC relate to assessing performance across a spectrum of thresholds, not performance at one particular threshold.
Consequently, your plan seems to be equivalent to ...
- 35.8k
2
votes
Accepted
Predicted Probability with XGBClassifier ranging only from 0.48 to 0.51 for either class
As @Adrian already pointed out, the reason the predicted probabilities are close to 0.5 is because the model has very small number of trees
'n_estimators': 15
and ...
- 36
0
votes
Accuracy Classification Result of Multiclass SVM
It is hard to give a qualified answer without the data, so I'll have to speculate:
$100 \times 100$ is a $10,000$-dimensional feature space (and that's if we're taking greyscale only). You have no ...
- 6,860
0
votes
Accepted
Why is $AUC=0.5$ and a 45-degree line for a ROC curve considered baseline performance?
When it comes to AUC, baseline performance means the performance of a model that has the same distribution of predictions for both classes.
From this, the diagonal line and $AUC=0.5$ follow naturally ...
- 35.8k
1
vote
Why is $AUC=0.5$ and a 45-degree line for a ROC curve considered baseline performance?
The AUC is the probability that a randomly selected positive is ranked higher than a randomly selected negative. So an AUC of 0.5 is the performance of a classifier that does not rank positives higher ...
- 81.3k
1
vote
How to *formalize mathematically* that a binary classifier has no predictive performance?
This is complicated, since supervised learning can have so many flavors, but a few general principles can lead you to solve special cases as they arise.
The first important topic to consider is what a ...
- 35.8k
2
votes
How do I combine arbitrarily-ranked lists?
From your example it seems that you want to weight the top ranks heavily. 500-1-1 is ranked higher in your view than 3-3-3, and even 1-500-500 should still be rated high.
What about this: Rank ...
- 13.5k
2
votes
Accepted
How to design cross-validation and testing scheme when N is small?
In addition to the excellent points in Camille's and EdM's answers:
Purpose of the model/study
Whether something sensible can be done or not depends crucially on the purpose of the modeling.
For any ...
2
votes
How to design cross-validation and testing scheme when N is small?
I understand that N is too small. Is it foolhardy to even attempt classification in this scenario?
Probably, at least as you propose to proceed.
First, for train/test splits to work well, you need on ...
- 69.1k
2
votes
How to design cross-validation and testing scheme when N is small?
Cross validation is not the only way to compare different models. Several model selection criteria, such as the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) can ...
- 1,707
0
votes
Importance for Color in X ray imaging for Detection of Pneumonia
transforming RGB to grayscale lead to loss of important information of colours
It would be easier to ask this to a radiologist instead of a statistician.
From the statistical point of view you try ...
- 51.8k
1
vote
Importance for Color in X ray imaging for Detection of Pneumonia
Are we talking classic X ray radiographs or some fancy modern technique ( MRI, PET-CT, Spectral-CT, ...). Classic X ray radiography has no color information. They measure how much of an electric ...
- 7,997
0
votes
How to find the confidence level of a classification?
Understand what you are modeling, talk to domain experts, collect relevant data. Accept that some things simply cannot be predicted with certainty. How to know that your machine learning problem is ...
- 101k
0
votes
What impact does increasing the training data have on the overall system accuracy?
I agree with @Serendipity:
The performance of neural networks can continually improve as more and more data is provided to the model, BUT the capacity of the model must be adjusted to support the ...
- 101
7
votes
Accepted
Does statistically simple algos qualify as AI algos?
There's no universally accepted definition of artificial intelligence. Since this directive came from your superiors, it does not matter whether anyone here considers the projects you've proposed to ...
- 330
1
vote
What is the statistical model for a multi-label problem?
You can think of the multinomial (aka multinoulli) distribution used in multinomial classification as a multivariate discrete distribution obtained in two steps:
assign a Bernoulli marginal ...
- 1,133
0
votes
Resizing input vectors of different lengths
If I understand your use case correctly, it is important to keep the time between the sensor measurements and the failure evaluation approximately comparable. So one approach could be to bin the time ...
- 8,694
3
votes
What is the statistical model for a multi-label problem?
Let's generalise your example by supposing that we have $K$ types of animal/object. As you correctly point out, if we are dealing with observations where these types are mutually exclusive then we ...
- 102k
0
votes
Accepted
Measuring response bias from a confusion matrix
The Python package sdt_metrics by Roger Lew implements several non-parametric response bias measures. Unfortunately, the package is not maintained, but the references are still useful.
One of these ...
- 1,717
3
votes
What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?
Find a good probabilistic model (by optimizing a proper scoring rule). Then vary your threshold to optimize F1 - this is a straightforward optimization problem, and bisection search will work well.
...
- 101k
2
votes
Accepted
How can area under ROC (AUC) be bad when precision, recall, and accuracy are all good?
Wrangling the numbers, I get the following as percentages of the dataset:
TP≈0.685037 FP≈0.288597 FN≈0.0164671 TN≈0.00989861
(So it seems you've swapped the positive class. That's not important for ...
- 3,251
0
votes
Why do we need natural log of Odds in Logistic Regression?
Why do we need natural log of Odds in Logistic Regression?
The log of odds is logistic regression by definition. So the question is more like, why do we need logistic regression? For this there are ...
- 51.8k
0
votes
Why do we need natural log of Odds in Logistic Regression?
In the equation:
$$
ln (p/1-p)= \beta_0 + \beta_1X
$$
The range of the right hand term is $(-\infty,+\infty)$ while without log the range of the left hand term $(p/1-p)$ is $(0,\infty)$.
Without log, ...
0
votes
Categorical data for binary classification
Yes, you can do it, but be careful with the meaning of your content. All data can change from fine-scale (numeric) to coarse-scale (binary) but it will lose some detail.
- 1
12
votes
Accepted
Is higher AUC always better?
AUC is a simplified performance measure
AUC collapses the ROC curve into a single number. Because of that a comparison of two ROC curves based on AUC might miss out on particular details that are left ...
- 51.8k
6
votes
Is higher AUC always better?
(observed) AUC can be influenced by statistical fluctuations
The ROC Curve is usually based on a sample of real world data and taking a sample is a random process. So there is some randomness in the ...
- 7,997
Top 50 recent answers are included