My questions is: Is there any particular converging pattern for Least Square Estimators and Maximum Likelihood Estimators of Weibull distribution when we increase the sample size? is there any mathematical explanation for that? To be clear, why the MLE estimator is deceasing (looks like exponential decrease) with sample size and converging to the true value (true value =0.5), and why the LSE is increasing with sample size and converging to the true value? (Apologize for a little bit tedious). I'm working on something about comparison of Least Square methods and Maximum Likelihood Estimation on Weibull Distribution. What I'm doing is that conducting a simulation study (Monte Carlo Simulation with 5000 iterations) to compare these two methods for different sample size (from 5 to 1000). The following figure indicate that for MLE by increasing sample size $N$, the estimated shape parameter is converging to the true value from a relative larger value than true value in a decreasing pattern. While for LSE, the estimated value start from a relative smaller value than the true value and then converge to the true value with a generally increasing pattern. The true values for the parameters are: shape parameter = 0.5 and scale parameter = 1.

Vertical axis is the estimated shape parameter, horizontal axis is sample size starting from 5 to 1000

The paper I'm working on can be found from here

Looking forward to any recommendations and comments. If my statement is not quite clear for you please let me know.

  • $\begingroup$ Please explain what you mean by a "converging pattern." It seems like all you need to know is whether this "least squares" method, whatever it might be, is consistent. Doesn't the paper examine that issue? $\endgroup$ – whuber Apr 26 '17 at 21:43
  • $\begingroup$ Thanks for the quick reply. From the above figure, as the sample size increase, the estimated shape parameter (take MLE as an example) is decreasing and converge to the true value (0.5). My question was is there any mathematical explanation for this pattern? Why for MLE method, the estimated shape parameter is decreasing with sample size, and for LSE, it is increasing with sample size? $\endgroup$ – keqiao li Apr 26 '17 at 21:53
  • $\begingroup$ That is called bias. You have found that for these data one of the methods is consistently biased high and other one is biased low. $\endgroup$ – whuber Apr 26 '17 at 22:01
  • $\begingroup$ @whuber Right, I agree with you. But how to explain this mathematically? $\endgroup$ – keqiao li Apr 27 '17 at 1:20
  • $\begingroup$ You compute the bias. That's a matter of getting the details of the estimators. Since you haven't provided those, it's hard to see how we could provide specific answers. $\endgroup$ – whuber Apr 27 '17 at 13:13

To compare the different estimators, we generally use the bias and mean square error (MSE) as indicators in the simulation. The values of likelihood function and sum of square of residuals cannot be used for comparing the estimators. About the increase of sum of square and decrease of likelihood along the sample size, they come from the definition of these two things. If you clearly understand the definitions, they are not problems.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.