Skip to main content
8 votes

What data are used to find the final threshold for a medical diagnostic test?

I have three thoughts. First, shouldn't you be sampling from a population that is similar to the one you will eventually be testing? This is unlikely to be a "normal" population. We ...
Peter Flom's user avatar
  • 124k
7 votes

What data are used to find the final threshold for a medical diagnostic test?

If you oversample the minority class, your model parameters and predictions will be biased. This is usually not a good thing. However, if data collection is expensive, it may be an avenue worth ...
Stephan Kolassa's user avatar
6 votes
Accepted

Interpretation of a decision tree plot

The answer to your first question depends on what, exactly, you mean by "easier". The answer to your second question is "no". The fact that A was found on the first branch simply ...
Peter Flom's user avatar
  • 124k
4 votes

Taking into account a non-symmetric loss function in a classification problem

I would recommend separating the modeling and the decision aspect: model probabilistically, assess these probabilistic classifications using proper scoring rules, and use thresholds when turning these ...
Stephan Kolassa's user avatar
2 votes

We have sensitivity-specificity space (ROC curves) and precision-recall space (PR curves, $F_1$ score). What work has been done with PPV-NPV space?

As you mentioned, some subsets of these metrics might be implicitly included in another chosen subset. Personally, I haven't come across much literature focused on the PPV-NPV space. I think it might ...
Johan de Aguas's user avatar
1 vote

Evaluating a classifier's performance on different groups of subjects

Since precision and recall are basically binary outcomes, you could compare the precisions and recalls directly. However, with the given sample sizes I think you're probably going to have a poor time ...
Demetri Pananos's user avatar
1 vote

How to interpret this confusion matrix in R?

The accuracy of your model is 0.82, but this isn't very good, as there is 82% of the data in one class. In other words, if you made no model at all, you could do just as well as your model. You can ...
Peter Flom's user avatar
  • 124k
1 vote

Proper usage of K-fold cross validation and finalizing model

Yes, when you do cross validation, you should not include the test data in this procedure. The test data is for your evaluation and reporting loss, accuracy, etc. computed on this set makes most sense....
picky_porpoise's user avatar
1 vote
Accepted

How to split and sample "Panel Data" when training a Logistic Regression to predict future outcomes

Best Practice for Splitting Panel Data For splitting panel data, the following approaches are often considered best practices: Temporal Validation (Out-of-Time) This involves setting aside data from ...
Esben Eickhardt's user avatar
1 vote

High Precision and High Recall issue- Random Forest Classification

You go into the modeling by telling the model to expect category zero. Category zero is, after all, considerably more prevalent. You then present evidence about class membership by giving the model ...
Dave's user avatar
  • 64.2k

Only top scored, non community-wiki answers of a minimum length are eligible