I have a random forest model that predicts a variable. This variable is not a categorical class but rather a number from 0 to 1. What is the best way to evaluate the accuracy of the generated models in this case?
Should I split the training and test parts and then simply calculate linear correlations between predicted and observed values in the test class?
Is there a more elegant solution? If so, is there an R package that implements this?
A standard way to evaluate model accuracy on continuous data is to compare the mean squared error (MSE) of your candidate models.
$$MSE = \frac{1}{n} \sum_{n=1}^{n} (y_i − \hat{f}(x_i))$$
The lower the MSE the better.
Section 2.2 of An Introduction to Statistical Learning is a good reference for this.
This and this will also be helpful for choosing metrics for evaluating your model's accuracy.
Linear correlation is probably not the best tool to use. After all the correlation between temperature measured in degrees Fahrenheit and degrees Celsius will have a high correlation, but not be good predictions of each other (unless close to -40).
More commonly the Mean Squared Error (MSE) or just a sum of squared errors (differences between the observed and predicted value) is used.
There are R packages that do cross-validation (modelr is one, I am sure that there are others).
For random forests, another common option is to use the out-of-bag predictions. Each individual tree is based on a bootstrap sample, this means that each tree was fit using on average about 2 thirds of the data, so the remaining 1 third makes a natural "Test" set for validation. For each observed data point, use the trees that did not include that data point to do the prediction and compare. The randomForest package does this internally for variable importance.
Besides measuring a correlation between label and prediction, you could first of all calculate Mean Squared Error on both train and test and compare them. You could also calculate the Mean Absolute Error. Or go with "Mean absolute percentage error". The choice should be based on your task.
I think the most commonly used goodness of fit measures are either R-square and (R)MSResiduals or (R)MSError. Regarding R packages: Most packages that implement random forests for regression also implement a way to evaluate the goodness of fit (see for example this tutorial)
