Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm using R tool to make machine learning algorithms. I made algorithm and now I have to evaluate model accuracy. So I used train(), trainControl() function. In trainControl() function, I used cv & repeatcv parameters to evaluate.
I am wondering that how to calculate model accuracy in cv & repeatcv. Is it calculated mean of produced algorithms?? or Is the model with the highest accuracy selected?
Have a nice day!
In order to evaluate performance of a model, you split your data into train and test datasets, use cross-validation to test the model on the train data. In the regression setting, the most commonly-used measure of accuracy of the model is the mean squared error (MSE): $$ MSE=1/n∑(y−f(xi))^2 $$ where $f(xi)$ is the prediction that you've got for $i$th observation.
For classification setting we compute error rate: $$ 1/n ∑I(yi \neq f(xi)) $$
There are other methods of accuracy measuring, you can easily google them.
