New answers tagged

0 votes

improving classification accuracy of the dataset as a whole by considering classifier distributions

If the problem is to estimate the relative class frequencies in a test set (or operational conditions) that may differ from those in the training set, then the best approach is to use a probabilistic ...
Dikran Marsupial's user avatar
0 votes

Appropriate feature selection for classification

I ended up generating many random predictors and seeing what their importance score was looking like. I got at the end a range of importance levels stemming from random variables, that I saw like this:...
Tino D's user avatar
  • 113
1 vote

Predicting the order/ranking of continuous observations

Don’t think of creating pairs in the data. Use an ordinal regression model for continuous Y, which requires no binning nor knowing how to transform Y. The proportional odds model provides the ...
Frank Harrell's user avatar
1 vote
Accepted

how does classError work in mclust package?

The documentation of mclust::classError explains that If more than one mapping between predicted classification and the known truth corresponds to the minimum ...
dipetkov's user avatar
  • 9,550
0 votes

How to define Precision when we have multiple predictions for each ground truth instance?

Clearly, there is no single correct evaluation protocol. It depends on what you aim for in your detection system. The properties of the detector that you want to quantify are typically determined by ...
Marko Lalovic's user avatar
1 vote

Neural Network Classification - targetting class probability and not the class themselves

A very simple way would be to just turn your problem into a "regression" one with squared loss. (I am using scare quotes because the distinction between regression and classification is, in ...
Stephan Kolassa's user avatar
2 votes

Neural Network Classification - targetting class probability and not the class themselves

Consider the binary cross-entropy loss $L = -y \log f(x) + (1-y) \log (1-f(x))$ with binary labels $y \in \{0,1\}$ and our model produces predictions $f(x)$ from the features $x$. If we $n$ ...
Sycorax's user avatar
  • 90.4k
2 votes

Linear algebra properties of a confusion matrix (eigenvalues, eigenvectors, and determinants)

The eigenvalues would really only reveal how many classes (single classifier) or how many classifiers are correlated with one another (multiple classifiers). But if you look at the quasi-diagonalized ...
Leif Peterson's user avatar
4 votes
Accepted

Can a ML classifier's prediction be understood as a probability?

That would be desirable, but it is not guaranteed to make as much sense as we might like. First, you could make an argument that any predicted $p(\mathcal C_k|\mathbf x_i)\in[0,1]$ is a probability in ...
Dave's user avatar
  • 60.9k
1 vote

What is a scoring rule for binary classification that is not dependent on the "difficulty" of classification?

I think you’re looking for a measure of calibration, which measures how your predicted probability values reflect reality. That is, when $\hat p=0.7$ is predicted, does the event happen $70\%$ of the ...
Dave's user avatar
  • 60.9k
0 votes

Estimating "prevalence" from a classifier's precision and recall?

Precision is a function of sensitivity/recall, prevalence, and specificity. $$\text{Precision} =\dfrac{ \text{sensitivity}\times\text{prevalence} }{ \text{sensitivity}\times\text{prevalence} + \left[ \...
Dave's user avatar
  • 60.9k
2 votes

Comparing classifications based on different features with McNemar test

McNemar is only for binary outcomes (0,1), (1,2) or test results (+,-), and thus two classes, not multiclass. A multiclass test for two classifers can be performed using Cohen's kappa. However, ...
Leif Peterson's user avatar
1 vote

Сlassifying clusters

The great benefit of unsupervised learning (e.g. clustering) is that you do not need labeled data. Therefore, just from the features of each data point and the distances between them, you can generate ...
LevG's user avatar
  • 183

Top 50 recent answers are included