What is the RMSE of k-Fold Cross Validation? I am testing a neural net  to predict numeric values. For that i am using a Training,Validation and Test split. I made a manual 4-Fold CV, this means i am getting 4 RMSE error, each one is the error of the i-th Fold on the test data.
How do i get global RMSE of all 4 Folds. Would it be (rmse_1 + rmse_2 + rmse_3 + rmse_4)/(number of all predictions)
 A: To be correct, you should calculate the overall RMSE as
$\sqrt{\frac{RMSE_1^2 + \dots + RMSE_k^2}{k}}$.
Edit:
I just got from your question that it may be necessary to explain my answer a bit. The $RMSE_j$ of the instance $j$ of the cross-validation is calculated as $\sqrt{\frac{\sum_i{(y_{ij} - \hat{y}_{ij})^2}}{N_j}}$ where $\hat{y}_{ij}$ is the estimation of $y_{ij}$ and $N_j$ is the number of observations of CV instance $j$. Now the overall RMSE is something like $\sqrt{\frac{\sum_j{\frac{\sum_i{(y_{ij} - \hat{y}_{ij})^2}}{N_j}}}{k}}$ and not what you propose $\frac{\sum_j{\sqrt{\frac{\sum_i{(y_{ij} - \hat{y}_{ij})^2}}{N_j}}}}{\sum_j{N_j}}$.
A: It's not a great reference, but in this notebook (look for cell starting with "Now let's compute RMSE using 10-fold x-validation") they add up the square errors (using a dot product) of all the predictions in all the cross validations, and then at the end divide by the number of predictions and square-root, i.e;
$\sqrt{\frac{1}{n} \sum_k{\sum_j{(y_{jk} - \hat{y_{jk}})^2}}}$
This makes the most sense to me, in the answer given by user1449306 the size of the folds would have an effect, which doesn't make sense?
To get to this from the list of RMSEs, they could each be squared and multiplied by the number of test points in each, then added together and divided by the total number of test points (and then squarerooted). Roland's comment is correct.
A: Here's my take. First we square the values, then we multiply with the counts, then we add them all up, divide by total count and take the square root. In pseudocode (Python), this could be implemented like
# Overall RMSE from list of rmses
new_list=[x*x for x in fold_wise_rmse]
counts  =[len(x) for x in val_sets]
nldotcounts=np.dot(new_list,counts)
overall_rmse=np.sqrt(nldotcounts/sum(counts))

where fold_wise_rmse and val_sets are both list structures
