In the calculation of RMSE, linear regression uses degrees of freedom(n-p) as divisor and neural network(feed-forward in my case) uses the total data number(does it have degrees of freedom as well?).
It is 'unfair' to use RMSE to compare which one has a better fit to the data response because linear regression alone suffers a 'punishment'. But RMSE has a clear definition to get an unbiased estimator in regression. If I use the 2nd formula for both models, I changed its definition for regression. Residual sum of square is comparable but it is a summation and doesn't show the deviation in single point level. Are there better ways to compare the result of linear regression and neural network?
I save the cross-validation problem for future. Only training data are discussed here.