6
Should predicting on your training set always throw out a very a high accuracy? Or would that indicate overfitting?
No, not always. The reason some models perform better than others is due to the bias/variance tradeoff. Logistic regression is a linear model, and hence is high bias. A random forest is capable of learning non-linear effects (low boas) of ...
3
People measure predictive accuracy on the test set and not on the training set for good reasons. If both models deliver 65% on the test set, I'd say they're both of pretty much the same prediction quality, and the fact that the Random Forest (RF) has a much higher value on the training set wouldn't bother me much. It does say that the RF overfits the ...
2
For some algorithms a bad initialization may matter and may be due to the particular random seed. In such cases, it may make sense to try to find a good initialitzation (=good random seed) that then leads to a good performance (or to find a way of modifying the training to reduce such effects). However, one should really be convinced that this is going on, ...
1
I see you have any questions. Q&A sites like this are usually not best suited for such cases, I'd encourage you to ask one question at a thread next time. Also given the number of questions, it'd be probably a good idea to start with one of many great handbooks on machine learning. Nonetheless, I feel like the questions are fairly easy to answer, so let ...
1
As @whuber points out in a comment, a 32-leaf tree may have depth larger than 5 (up to 32). To answer your followup question, yes, when max_leaf_nodes is set, sklearn builds the tree in a best-first fashion rather than a depth-first fashion.
From the docs (emphasis added):
max_leaf_nodes : int, default=None
Grow trees with max_leaf_nodes in best-first ...
1
Version 0.24.2 is taking this use case into account: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier
As specified in the documentation, the shape of the target you're passing to the fit function is like (n_samples, n_outputs).
Only top voted, non community-wiki answers of a minimum length are eligible