New answers tagged

0 votes

ROUGE-N for multiple references

I was reading the ROUGE paper right now and I came across with the same doubt, so I was hoping to find someone else with the same question. I agree with you: in the first formula the summation over ...
user avatar
  • 1
1 vote

Multiclass Unbalanced Classication :Very very low F1 scores, high precision low recalls

" I want the model to catch the positives and neutrals and not misclassification on negative class (aka very small False Positive on Negative class I suppose). " The results you have are ...
user avatar
2 votes

Best hyperparameter is not consistent among different seeds

I can not find an optimal hyper-parameter for all seeds That's the stochastic nature of machine learning. We don't have all the data, so any inference/training is vulnerable to noise. The model ...
user avatar
  • 52.5k
2 votes

overfitting of random forest in r

How does the 70 % accuracy of your CV compare to the rF's oob estimate? The behaviour you observe is to be expected for random forests, see also my old answer: https://stats.stackexchange.com/a/...
user avatar
0 votes

Calculating "accuracy", "recall" etc. without classification

Some observations, now I've had time to think about what I did. While I want to evaluate the model itself, rather than the decisions it will be used for, at the time of decision-making there will be a ...
user avatar
1 vote

Calculating "accuracy", "recall" etc. without classification

I think the main problem is that you will get the same problems as for the underlying KPIs, just in a probabilistic flavor. For instance, let's assume that conditional on your predictors, a given ...
user avatar
1 vote

overfitting of random forest in r

Random Forest Classifiers (RFC) with 100% training accuracy are not necessarily problematic. Make sure you are optimizing your hyperparameters on a separate validation set, this is especially ...
user avatar
1 vote

Finding optimal F1 threshold for classifier without probabilities (e.g. SVM)

Yes, SVM is NOT a probabilistic estimator, BUT see docs: it has param probability bool, that is by default =False. Just put it True to enable probability estimates. This must be enabled prior to ...
user avatar
  • 49
0 votes

precision recall breakeven point

Precision and recall measure the performance of a set of items which are predicted to be positive. The break-even point measures the performance of a ranking of items which puts the items most likely ...
user avatar
  • 721
0 votes

Is it disingenuous to test tree-based regressors on training data for the sake of comparison to a linear model?

As to the titular question, yes. We know that tree based estimators are low bias and potentially high variance. It would be absolutely no surprise that the training error is smaller for the tree ...
user avatar
4 votes
Accepted

What is the opposite of precision called?

That would be called the negative predicted value. See wikipedia
user avatar
  • 124
0 votes

Grid search with cross-validation performing worse on test set than baseline model

I think is possible with cross validation the is getting a better model. Try to see the performance of your baseline model along the cross validation folds. And it is somewhat overfitted for the test ...
user avatar
  • 74

Top 50 recent answers are included