I am using about 256 predictors and target is sales. I am using a software called Alteryx which is R based. I have tried to run Random Forest, Spline model and Neural nets on same data.
I used partitioning to create test and training data sets.
I am fairly new to field, hence I am not sure how to compare performance of these models.
I was advised to compare Pearson correlation coefficient between predicted and actual value for all models and select the best one.
Although I am new, from what I have learned, I don't think its a good way to compare models.Pearson for Forest model is about 0.98 and for neural nets it's about 0.91. both figures are for testing set.
Is there any better way to compare and cross validate models?
Should I use R squared value/ AIC to compare?Is there any other manual way to compare models?
I have predicted and actual values for training data sets for all models, is there any way to compare using those?