# Calculation of RMSE and $R^2$ to compare five numerical methods for determining Weibull parameters

I have calculated Weibull parameters namely, shape parameter and scale parameter using five numerical methods: graphical, moment, maximum likelihood, empirical and energy pattern factor methods. Since I need to compare these methods to find out best method, I need to calculate RMSE and $R^2$ for each method.

But even if I know the formulas for RMSE and $R^2$, I am not being able calculate as I am confused and a bit unfamiliar with the terms used in those formulas. Please refer the following research journal and I want to clearly understand about 'Prediction performance of Weibull distribution model' section of the journal.

My queries regarding 'statistical tests' are:

1. How do you calculate 'frequency of weibull' or sometimes it says 'predicted data' ?
2. To calculate RMSE and $R^2$, do we use frequency of the data ( observed and predicted) or actual data (e.g wind speed, temperature)?
• If the "best" estimation method is the one that gives the lowest root mean square error on the sample, then why not use parameter estimates that minimize RMSE? Might be useful to read up on what properties define a good estimator in general. – Scortchi - Reinstate Monica Dec 15 '14 at 16:26

Let's decipher formulas in the paper:

$$RMSE = \left[ \frac{1}{N}\sum^N_{i=1} (y_i - x_i)^2 \right]^{1/2}$$

this one is pretty clear: RMSE is a square root of averaged square difference between $y_i$ and $x_i$ where $y_i$ is an observed frequency and $x_i$ is a frequency predicted by the model.

The formula for $R^2$ they provide is more complicated:

$$R^2 = \frac{ \sum^N_{i=1} (y_i - z_i)^2 - \sum^N_{i=1} (y_i - x_i)^2 }{ \sum^N_{i=1} (y_i - z_i)^2 }$$

It is ambiguous since authors describe its parameters as "Where: $y_i$ is the actual data, $x_i$ is the predicted data using the Weibull distribution, $z$ is the predicted data using the Weibull distribution, $N$ is the number of observations", so $x$ and $z$ seem to mean the same thing and it would not make sens. However, they quote other paper they wrote that describes $z$ as an mid value of $y$.

• i want to quote one of your sentence here: '...xi is the predicted data using the Weibull distribution...' . How exactly am i going to do prediction of data using weibull distribution???? This is the major problem i am having right now. @Tim – kancha Dec 16 '14 at 6:21
• You plug in the data into the formulas and get predicted frequencies. – Tim Dec 16 '14 at 7:26
• Plug in the data into which formulas?? Is it Weibull PDF or CDF?? How to get predicted frequencies? An example would be great! @Tim – kancha Dec 17 '14 at 4:12
• But you have multiple examples in the article you quoted and the references it provides (e.g. the one I link in my response). Read those carefully and you'll find your answer there. – Tim Dec 17 '14 at 7:24