3 votes

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?

Find a good probabilistic model (by optimizing a proper scoring rule). Then vary your threshold to optimize F1 - this is a straightforward optimization problem, and bisection search will work well. ...
2 votes
Accepted

On using the same tokenizer for train and test data

The point of using a test set is to validate your model on unseen data. To do this, you need to apply exactly the same preprocessing and prediction function to the test set. If you used different ...
  • 121k
2 votes
Accepted

Predicted Probability with XGBClassifier ranging only from 0.48 to 0.51 for either class

As @Adrian already pointed out, the reason the predicted probabilities are close to 0.5 is because the model has very small number of trees 'n_estimators': 15 and ...
  • 36
1 vote

How do elections prediction works

so for one thing the variability only decreases with the square root of the number of people by the law of large numbers - so there is a diminishing returns effect on adding more people to the sample ...
  • 4,548
1 vote
Accepted

Survival analysis - risk-set based on current prediction time?

I would like to rank individuals based on their risk at the prediction time point. If the proportional hazards (PH) assumption holds, the time-varying aspect of the Cox model is time-varying ...
  • 69.1k
1 vote

How to represent the interval or uncertainty on regression predictions in an 'experimental vs predicted' plot?

You are close to the usual approach in your second last plot. I would say that the convention is to place the observed (i.e. experimental) values on the X-axis and the fitted values on the Y-axis, ...
  • 14.6k
1 vote
Accepted

How to define survival time (Start and end) for prediction model using cox regression in my dataset

Is there supposed to be a specific cutoff usually for the end of follow-up, or is it just what we're used to seeing for studies with a specific end time? There is no need to have "a specific end ...
  • 69.1k
1 vote
Accepted

R: weird behaviour in linear regression

The reason you are having this issue is because the name of your predicted variable from the original regression and the name of your predicted variable in the new data frame don't match. You can try ...

Only top scored, non community-wiki answers of a minimum length are eligible