I've got two basic questions for machine learning as newcomer.
- Cross-validation vs Test. I've heard people or online said it's expected that cross-validation data accuracy score is always higher than test data?
Is that true? To some extent it makes sense because there's always certain degree of overfit for training data. But my own experience is always two scores are similar, for example, 0.82 and 0.83
- What metrics should we look at for model evaluation? What I usually do is: Accuracy score for classification and RMSE for regression.
Should we do accuracy score for regression?