I am trying to understand over-fitting. I am using a regression tree method in Matlab. The given sample size is 500, which I divide into a training set of 400 and a test set of 100. I create the model for the training set and get a corresponding $R^2$ of about 84%. i.e. the in-sample $R^2$ is 84%.
When I use the model to forecast for the 100 in the test set, I get a negative $R^2$ which indicates a very poor model on out-of-sample data.
So, does it mean that my model over-fitted the in-sample data of size 400 and got a very high R-square? The regression tree in Matlab has pruning turned on by default so that should avoid over-fitting.
Any ideas as to what this situation might mean ?