Training RMSE is almost 5 times as high as test RMSE - need help understanding why I am currently trying to model the median_house_value (MHV) from a California data set from the 90's. But the training RMSE (230142) is almost 5 times as high as test RMSE (46371), could anyone help me to understand what I did wrong please? (I am new so please forgive me if I am asking the question wrong)
I will try to summarize what I did; please let me know if there is anything else needed to analyse the problem. You can find the full Python script in the github link at the bottom.
This is what the target variable median_house_value for training (y_train) looks like. I am almost certain that this is a right-censored target variable, as there is too many MHV>500k. (The median is 'raw data', in the sense that median_house_value comes with the dataset and is intended as the target variable)
count     14448.000000
mean     206661.314023
std      114948.004583
min       14999.000000
25%      120000.000000
50%      180500.000000
75%      264100.000000
max      500001.000000


What I did:

*

*Boxcox transformed y_train and y_test

*Bucketed longitude and latitude, and created a boolean feature indicating which bucket of long and lat the house is in (long/lat buckets are later dropped)

*Ordinal encoded the categorical feature 'ocean_proximity'

*Created a feature by crossing the amount of rooms and bedrooms

*Created polynomial features out of the top 5 most correlated features

*Boxcox transformed skewed features

*Scaled features using StandardScaler()

*Created geospatial features

*Used VotingRegressor based on XGBRegressor and RandomForestRegressor (optimised using RandomizedSearchCV)

*Used inv_boxcox to revert transformation on prediction, censored any MHV>500k, then calculated RMSE

This is the RMSE function I used:
def censor_inv_boxcox_rmse(X,y,model,lamb):
    pred=inv_boxcox(model.predict(X),lamb)
    pred=np.where(pred>500000,500000,pred)

    mse = np.mean(((pred - y)**2))
    rmse = np.sqrt(mse)
    return rmse

The best RMSE I have seen on this dataset is around 40000, but that one likely involved >400 features.
I tried changing the random_state of train_test_split and rerun the whole script, but the RMSE difference did not change.
The full script of what I did:
https://github.com/Pixy33/California-House-Price-modeling/blob/main/California%20House%20Price%20prediction%20model.ipynb
 A: Consider this an extended comment, not an answer.
Your question is why the training residual mean squared error (RMSE) is five times the test RMSE.
The challenge is that you have a 10-step procedure to build the model. There might be an issue in one or more of these steps. It's hard to know since you evaluate the performance once at the very end.
Here is an alternative approach. Build a very simple baseline model, with as few steps (lines of code) as possible. Maybe a decision tree or a random forest, omitting the more challenging features. Do you observe TrainRMSE >> TestRMSE? If yes, then you know the issue is with (how you load) the raw data. If no, then add in one of your data-processing/model-building steps. Repeat until you achieve good performance. During this exercise you may stumble upon the original problem. But even this doesn't happen, you would have developed a useful model.
Finally, an unrelated comment: Why are you looking at the test dataset while you are building the model? Split the training dataset into a subset for training and a subset for validation.
A: After a quick review of your analysis, the biggest thing I would point out is the thing that you already noted, you are treating the censored home values at \$500K as if they are numbers instead a censored value or categorical value.
Two paths forward that are commonly used (there are also others)

*

*Censored data analysis:  Accelerated failure model or Cox Proportional Hazards model that correctly utilizes the censored information.

*Two stage model: First regress whether the home price is greater than or less than \$500K, then regress the value below \$500K.  Ensure that both models are trained jointly and any hyperparameter tuning is done jointly.

