Actually, I didn't understand what is the uncertainty mean.
Maybe think like this: You try to measure predictive performance, and like any other measurement result, also your results are subject to random and systematic uncertainty.
Imagine for a moment that you had only 5 such predicted/reference pairs, or 500. Clearly, the RMSE based on only 5 pairs (= tested cases) is less well known, i.e., more uncertain, than an RMSE based on 50 or 500 pairs.
Now, RMSE contains several "sources" of prediction error: bias, i.e. the predictions being systematically off (which does not enter R²), and several sources of random error, i.e. variance.
Two of the latter include the variance uncertainty caused by testing only a limited number of cases (50) - that's the variance I explained above.
In addition, there is potential variation between your surrogate models: they may not yield totally stable predictions. I.e., two such models may yield different predictions for the same cases (imagine you acquire an additional case and let it be predicted by all 50 models).
Unfortunately, leave-one-out cross-validation leaves you with surrogate model and test case being collinear, so you cannot distinguish between the two. k-fold cross-validation, and even better repeated k-fold CV, would allow to separate these two sources of variance.
In addition, how and which important sources of variance enter the uncertainty of your RMSE estimate depends on your precise application scenario. In particular, on whether you use the cross validation as approximation to measure the RMSE of the training algorithm for data such as yours, or whether you approximate the RMSE of the actual model you obtain on all data points for production use. As you see, a quantitative discussion of these sources of uncertainty on your model performance estimate is quite difficult.
There is also some bias expected in the RMSE estimation via cross validation. If all is well (in particular with your splitting procedure), this should be small and pessimistic.
What you can and should do, though, is to discuss which of these sources are relevant in your case.
You can also have a jab at the sample/surrogate model instability based on jack-knifing (i.e. in turn leaving one pair out of the RMSE calculation) or bootstrapping your RMSE (or R²) calculations. However, this will typically not include all relevant sources of uncertainty on the model performance estimate.
It does give you a low guesstimate, though.