Evaluating mean-squared error Hello I am running a Regression Tree experiment.
I am new to Regression Trees, and I am using Mean Squared error to test my tree.
I am confused because I am getting a large Mean Squared Error but I am not sure how to evaluate if it is too high.
Should different successful trials expect varying MSE?
Do there exist methods for checking this?   
 A: I don't know for sure whether or not there is something wrong with your numbers/code, but if your error is really high, this would actually be pretty standard when you're working with isolated regression trees. 
If your error really is high though, try gradient boosting. The algorithm is a subset of boosting algorithms (look into AdaBoost too; it's in the video below). It essentially fits more trees to the residuals of your other models. Check out this video by Prof. Trevor Hastie of H20.ai (& Stanford): https://www.youtube.com/watch?v=wPqtzj5VZus
Hope this helps. Also, if you want to excel, read through this paper by Prof. Robert Schapire and Prof. Yoav Freund, founders of AdaBoost: https://cseweb.ucsd.edu/~yfreund/papers/IntroToBoosting.pdf It isn't about gradient boosting, but maybe you will find AdaBoost or LogitBoost to be more effective. 
A: Mean Squared Error provides SQUARED Values, of course they're large! If you use Mean Absolute Error, those values would be far more little. (In MSE, the higher is the diference between y_pred and y_true, the higher is the error, because it's squared. You punish more the higher mistakes than using MAE.)
For example, if your problem is about classification, the accuracy is a great measure to understand how far you're from the better scenario (100% accurracy) and the worse (0% accuracy). But with regression, there's no "maximum error", although there's minimum (0).
The thing is, unless your error is a comprehensive method of how much you're mistaken, you have to use Mean Squared Error as a COMPARATIVE METHOD.
So, if you test 2 different models (no matter which ones, it could be two different trees), the model that you would keep is the one with the lower error.
One of the approach that you couuld take to check when it's "too high" is to evaluate a naive model (all predictions are the mean, or random values...). With this, you can check if your model is clever than a random decision or naive decision model.
In addition, Mean Squared Error is a mean (a summary of your distribution of errors). I strongly suggest complement the calculus of the MSE with an histogram of your errors (that will allow you to see which observations are worse predicted, perhaps your error is correlated with a variable, or some outliers are making you have a higher MSE. Remember, mean is sensible to outliers.)
A: 
I am confused because I am getting a large Mean Squared Error but I am
not sure how to evaluate if it is too high.

This is up to you to decide. MSE is useful to compare your method with other methods.
