Is altering predictions before model evaluation allowed? I am doing analysis on a data set regarding tree volume prediction. I'm using regularized least squares as my prediction model and I'm using RMSE and cross-validation to evaluate my model. 
Currently, I have simply used cross-validation for selecting the model parameters and RMSE for evaluating the performance of the model, i.e. I have calculated the predictions $\hat{y}$  of my model and compared them with the true values $y$ using RMSE. 
Before calculating the RMSE-value I did not transform the predictions $\hat{y}$  in any way. What I mean by this is that if my model gave negative predictions $\hat{y}$, I simply plugged these negative values into RMSE-formula even though negative values don't make sense for tree volume. 
My colleague told me to first transform negative predictions to 0 and then evaluate the model. 
My question is:
is the transformation of negative predictions in this case allowed before calculating the RMSE-value? 
This transformation bugged me for some reason, because it seemed to me that I'm evaluating a different model than which I trained. Is my concern valid?  
 A: It is perfectly valid
Simple reason: you are using an algorithm which does something and outputs something. The only reason why you trust it is because you are using a metric to test the algorithm. It is completely up to you to implement your own algorithm. If it outperforms the previous one, take the new one.
And by "your own algorithm" I mean "the old algorithm with all weights below zero set to zero". You just add another "rule" to the algorithm. No problem with that.
To broaden the view: Did you already consider to use other algorithms? They may outperform your current one. And did you do the other things right? Enough data, not overfitting etc.? Because negative predictions hint to something wrong with your algorithm.
To summarize: You take the best performing algorithm. What the algorithm looks like (so if you add another transformation to a pre-existing "black box") is up to you. Just use cross-validation (plus a hold-out set). But may consider looking at other algorithms as they are probably better then your current with additional transformation. Just testing will give you the answer.
