# Tag Info

4

These models -- random forest, xgboost, etc -- are extremely sensitive to the hyper-parameter configurations, so there's no reason to believe that these hyper-parameters will yield good models. For xgboost, the number of trees and the learning rate are two examples of hyper-parameters which require tuning. Both have a strong effect on the model. Also, your ...

3

@Sycorax is very capable, so he is technically quite correct. This answer is more of an elaboration of a comment that supports his main assertions. Disclaimer: This is a very weak "tuning" so while it shows the concept it is nowhere near optimal, and will pretty strongly over-estimate the number of trees you need. I have thought that the Gradient ...

3

This is definitely possible, and not strange at all. Recall how accuracy and the F1 score are defined: $$\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\quad\text{and}\quad \text{F1}=\frac{2TP}{2TP+FP+FN}.$$ Now, probably the simplest possible way your F1 score can be greater than your accuracy is if you have just two observations, one TRUE and one FALSE. ...

3

You are performing classification by pixel, so a 768x768 is in fact 589824 samples. The number of support vectors might as well be a bottleneck, if you have many of them, because you must build a $n_\text{sv}\times n$ kernel matrix, which is huge.

2

You could do the oversampling outside/before the cross validation iff you keep track of the "origin" of the synthetic samples and treat them so that no data leak occurs. This would be an additional constraint similar to e.g. a stratification constraint. This is possible e.g. by doing a cross validation on the real-sample basis and inside the cross ...

2

The issue here is very likely to be the number of support vectors - if the fraction of points misclassified is just 5% then in your case you will have at least 29K support vectors. There are several different techniques you can use to achieve a sparser solution - see for example here or here (amongst many others). A simple alternative technique you can use (...

1

Yes. Because you have a fairly small amount of features already (15), it makes sense if you weren't able to reduce the dimensionality much further without reducing the explanation for variance. PCA is often done on datasets with hundreds or thousands of features to reduce the dataset. Although, to note, if you did have highly-redundant features, it is also ...

1

To add a separate dimension to my previous answer - there are two aspects which can impact the evaluation speed for prediction. The number of support vectors - this gives the number of kernel evaluations you need to do. The cost of evaluating the kernel function - which in this case is directly tied to the dimensionality of the problem, and the performance ...

1

I'll bring over my comments from the Stackoverflow question. (1) RF outputs are votes, not proportions. My advice is to try to set parameters for RF or just use a different kind of tree or different model altogether to get at least proportion of positive examples in each leaf. Starting from votes is just setting your starting position farther away from the ...

1

Internally sklearn uses scipy.linalg.lstsq function for finding solutions to linear equation, which is the same as numpy.linalg.lstsq, and its innerworkings are described in this post: How does NumPy solve least squares for underdetermined systems?

1

GridSearchCV in general performs cross-validation (by default, 5-fold), and (by default) selects the set of hyperparameter values that give the best performance (on average across the 5 test folds). It (by default) uses the estimator's score method to evaluation performance on the test folds. In the case of KernelDensity, score gives the log-likelihood of ...

1

After examining the problem more I convinced myself that the official way to do it is the right way. My objection was originally that doing regression against {0, 1} could result in jagged results. But actually, that's the basis for logistics regression! Isotonic regression is not fundamentally different in that sense. Here's a bad drawing to explain why ...

Only top voted, non community-wiki answers of a minimum length are eligible