Abstract example. Let's say, I want to compare MLE (Maximum Likelihood Estimation) and LSE (Least Squares Estimation) performance on the small samples for simple linear regression.
My idea: to generate two normally distributed variables x and y with very large number of observations. As far as I know, MLE and LSE are asymptotically equal, so they should return almost equal parameter estimations on the whole dataset.
Then I will take n random samples of size m (e.g. 1000 samples for each m from 10 to 1000) from the "population" that was generated on the previous step, build two (MLE-based and LSE-based) regression models on each sample and compare "real" parameter values and parameters from each model.
If for one of the methods (MLE or LSE) the mean of estimated parameter on the small samples will not be equal to the "real" parameter value, can I say, that method is giving biased estimations on the small samples?
If for one of the methods the variance of estimated parameter will be greater than for the other, can I say that the method gives less stable estimations?
How can I calculate the significance of this differences? Is this design valid at all? Maybe there are better alternatives?