From literature I understand that the desirable properties of statistical estimators are
- Unbiasedness - we want the estimator to give the correct parameter value theta, on an average, irrespective of the sample size--defined by
- Consistency - we want larger sample sizes to give progressively better estimates of the correct parameter value theta and asymptotically converge to theta in probability--defined by
- Efficiency - we want the unbiased estimator to have the lowest possible variance--as determined by the Cramer-Rao bound. Efficient estimators, however, need not exist in all situations.
I don't really understand each of these properties and the difference between them. Please explain the intuitive meaning of these properties followed by the math behind it.
Refernce Link - https://www.cs.utah.edu/~suyash/Dissertation_html/node6.html