I am working on an xgboost model using caret. I'm using cross validation, but don't know if I'm understanding it correctly. As I understand, it creates multiple training and test sets. Does this mean that the data doesn't have to be splitted to training and test sets before the modeling?
If so, how do I obtain the mean absolute errors from the test set? I am using R.
I tried to get them using the following code:
postResample(pred = pred, obs = obs)
but the MAEs seem rather optimistic. Is it possible to obtain the MAEs just for the test set?
I am also using hyperparameter tuning, if that matters. The model is also linear.