Questions tagged [efficiency]

A measure of the quality of a statistical estimator.

Filter by
Sorted by
Tagged with
57
votes
7answers
9k views

Examples where method of moments can beat maximum likelihood in small samples?

Maximum likelihood estimators (MLE) are asymptotically efficient; we see the practical upshot in that they often do better than method of moments (MoM) estimates (when they differ), even at small ...
19
votes
2answers
2k views

For what (symmetric) distributions is sample mean a more efficient estimator than sample median?

I have labored under the belief that the sample median is more robust measure of central tendency than the sample mean, since it ignores outliers. I was therefore surprised to learn (in the answer to ...
19
votes
2answers
4k views

Why $\sqrt{n}$ in the definition of asymptotic normality?

A sequence of estimators $U_n$ for a parameter $\theta$ is asymptotically normal if $\sqrt{n}(U_n - \theta) \to N(0,v)$. (source) We then call $v$ the asymptotic variance of $U_n$. If this variance is ...
13
votes
3answers
3k views

Why is the asymptotic relative efficiency of the Wilcoxon test $3/\pi$ compared to Student's t-test for normally distributed data?

It is well-known that the asymptotic relative efficiency (ARE) of the Wilcoxon signed rank test is $\frac{3}{\pi} \approx 0.955$ compared to Student's t-test, if the data are drawn from a normally ...
9
votes
1answer
3k views

Relative efficiency of Wilcoxon signed rank in small samples

I have seen in published literature (and posted on here) that the asymptotic relative efficiency of the Wilcoxon signed rank test is at least 0.864 when compared to the t test. I have also heard that ...
9
votes
1answer
599 views

When can't Cramer-Rao lower bound be reached?

The Cramer-Rao lower bound (CRLB) gives the minimum variance of an unbiased estimator. One sentence in the wiki page says "However, in some cases, no unbiased technique exists which achieves the bound....
9
votes
2answers
1k views

Is OLS Asymptotically Efficient Under Heteroscedasticity

I know that OLS is unbiased but not efficient under heteroscedasticity in a linear regression setting. In Wikipedia http://en.wikipedia.org/wiki/Minimum_mean_square_error The MMSE estimator is ...
7
votes
1answer
220 views

If $\operatorname{Var}\left(\epsilon_i\right) = h\left(X\right) \neq \sigma^2$, what can we know about $\operatorname{Var}\left(\hat{\beta}\right)$?

This question uses the derivations found here. The short version Consider a regression model. If the error variance is a known function of the data (rather than a constant), under what conditions ...
7
votes
1answer
287 views

Why efficiency matters?

Suppose we are trying to estimate the quantity $\theta$ and we have that the estimator $\hat\theta_n$. Suppose it is efficient, i.e. is variance is the smallest among certain class of other possible ...
7
votes
1answer
178 views

Is $X_{(1)} + X_{(n)}$ a good estimator for $\theta$?

Problem 8.7 From Van der Vaart's Asymptotic Statistics: Given a sample of size $n$ from the uniform distribution on $[0,\theta]$, the maximum $X_{(n)}$ of the observations is biased downwards. ...
7
votes
1answer
243 views

Efficient Estimator from Insufficient Statistic

Suppose that I have a statistic $T(X)$, and I know for sure that it is not sufficient to estimate a parameter $\theta$. Is it still possible to have an estimator $\hat\theta(T(X))$ that is efficient (...
6
votes
1answer
313 views

Seasonal adjustment a la Hyndman

Rob J. Hyndman has two posts here and here on forecasting weekly series. He suggests using a regression with ARIMA errors, $$ y_t = a + \sum_{k=1}^K (\alpha_k \ \text{sin}(2\pi k t/m)+\beta_k \ \...
6
votes
2answers
447 views

How to compute efficiency?

Suppose instead of maximizing likelihood I maximize some other function g. Like likelihood, this function decomposes over x's (ie, g({x1,x2})=g({x1})g({x2}), and "maximum-g" estimator is consistent. ...
6
votes
3answers
2k views

How can one speed up this correlation calculation in R without multicore?

I have a colleague who calculates correlations in which one set of scores for a subject (e.g. 100 scores) is correlated with another set of scores for that same subject. The resulting correlation ...
6
votes
2answers
2k views

Is MLE more efficient than Moment method?

I have got some small data sets (about 8 to 11 data points for each set), following Normal distribution. I would like to find out the 95% confidence interval of the 0.005 and 0.995 percentile of each ...
5
votes
2answers
1k views

Increase training performance of a neural network with low learning rate?

I am trying to train an Artificial Neural Network for classification. In the input layers, I have 402 neurons; the first 400 are binary, and the last two are floating points in the range -1 to 1. In ...
5
votes
1answer
180 views

Efficiency of Bayesian Estimator

In the Wikipedia article for Minimum Mean Square Error, the Bayesian estimator is referred as "asymptotically efficient" using the similar arguments of Fisher information from frequentist statistics. ...
5
votes
0answers
447 views

How general is the backfitting algorithm?

Hastie \& Tibshirani's original approach to fitting generalized additive models was the backfitting algorithm. For a model of the form $$ y = \alpha + \displaystyle\sum_k f_k(x_k) + \epsilon $$ ...
5
votes
0answers
110 views

Relation between asymptotic relative efficiency for tests and estimators

The asymptotic relative efficiency for unbiased estimators is the limit of the ratio of the variances as the $n\rightarrow \infty$. Is there a relation to asymptotic relative efficiency according to ...
4
votes
2answers
2k views

Efficiency of beta estimates with heteroscedasticity

I need something clarified and that is when you have non-constant variance, estimates won't be biased but will be a problem when it comes to the S.E. formulas and efficiency. Therefore OLS estimates ...
4
votes
3answers
993 views

The concept of efficiency

I have some problems in understanding the concept of efficiency as related to an estimator. My sources (Mukhopadhyay, 2000 and Casella, Berger, 2002) do not treat this argument as I expected since ...
4
votes
3answers
323 views

How to improve the efficiency of an estimator when the estimates significantly vary among subgroups?

In a clinical trial, patients enter the hospital (hence, the study) at different time points. We have observed that patients who respond to the primary therapy early survive longer than those who ...
4
votes
2answers
115 views

ARMA Estimation Efficiency of the Mean

Usually in books related to ARMA Time Series, it is assumed that the series is 0 mean. If not, the recommended standard procedure is to simply subtract the sample mean from the series and continue ...
4
votes
1answer
1k views

Restricted OLS have less variance than OLS?

According to Gauss-Markov Theorem, ordinary least squares (OLS) is the best linear unbiased estimator (BLUE). How then can restricted OLS have less variance? Please tell me the reason.
4
votes
3answers
2k views

Efficiency and the number of regressors?

An econometrician told me that I shouldn't keep adding new variables to the model even if I have reason to believe they're relevant to the response variable, as it "reduces the efficiency of the other ...
4
votes
1answer
488 views

Efficiency of GLS over OLS when regressors are not fixed

Suppose we have a regression model $$ y=X\beta+u,\quad E(u)=0,\quad E(uu')=\Sigma. $$ Let $\hat\beta$ and $\bar\beta$ respectively denote the OLS and GLS estimator. Then, when $X$ is fixed (or when $X$...
4
votes
1answer
4k views

OLS Regression : Efficiency of the estimator of the variance of the residuals under the assumption of normality

My question is probably already answered somewhere but I did not find it. In the standard linear regression model under the assumption that residuals are normally distributed, we have a result ...
4
votes
2answers
234 views

A Primer on Estimator Efficiency?

Can someone recommend a text with derivations of classical estimator efficiency results? I'm particularly interested in likelihood and pseudo-likelihood estimators for multi-variate discrete models
4
votes
1answer
66 views

Which is a better estimator, averaged functions vs. A function of an average?

Problem: Assume that we want to estimate $f(\theta)$ with a pre-specified strictly increasing function $f$ and a parameter $\theta$. Let $\hat{\theta}_1$ and $\hat{\theta}_2$ be unbiased estimators ...
4
votes
0answers
127 views

Minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed

What is the minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed? median When we wish to estimate the median, $\mu$, of a normal distributed variable then ...
4
votes
0answers
169 views

Is an inadmissible estimator necessarily dominated by some admissible estimator

Basic example: $X$ has a $p$-variate iid standard Normal distribution; the sample mean is not admissible if $p>2$ and is dominated by the Stein shrinkage estimator. However, the Stein shrinkage ...
4
votes
0answers
909 views

Are there efficient estimators for the variance of an exponential family?

Let us consider the Gaussian model $\mathcal{N}(\mu,\sigma^2)$, where both $\mu$ and $\sigma$ are unknown. I have learnt that (for example, from Amari's information geometry book) the exponential ...
3
votes
4answers
7k views

How to speed up training of a Neural Network?

I'm writing a thesis where I developed a script that generates NN and precalculates weights and biases to reduce a required number of epochs when I train a network. In my work, using examples I ...
3
votes
2answers
47 views

Vector-valued estimators, intuitively why $var(\widetilde{\beta})-var(\widehat{\beta})$ being p.s.d. means $\widehat{\beta}$ more efficient?

For two scalar unbiased estimators $\widehat{\alpha}$ and $\widetilde{\alpha}$, we know that if one has smaller variance, then we say it is more efficient, which intuitively means that this estimator ...
3
votes
1answer
147 views

What may be an inefficient estimator of the population mean?

If the sample mean is an efficient estimator of the population mean, what may be an example of an inefficient such estimator?
3
votes
1answer
1k views

Estimating VAR by GLS versus OLS: efficiency

Suppose I have a VAR model with different regressors in different equations (this could be due to restricting some coefficients of a full VAR($p$) model to zero or having some different exogenous ...
3
votes
1answer
2k views

Bootstrapping data envelopment analysis efficiency score using R

I want to perform bootstrapping for calculation of efficiency score from data envelopment analysis (DEA) using R. Are there any examples of data and results for this type of analysis in R to enable ...
3
votes
1answer
62 views

In the case of lognormal distribution, when can the median be more efficiently estimated than the mean?

I understand, that in the case of normal distribution, the estimation of the mean (from samples) is more efficient (i.e. of less risk), than the estimation of the median. According e.g. to this post, ...
3
votes
1answer
1k views

Biased and Efficient estimators

Is unbiasedness a necessary condition for an estimator to be efficient? For example, if $\hat {\theta}= \frac{\sum_i^n X_i}{3}$, I assume $\hat {\theta}$ can't be efficient in a Cramer-Rao lower ...
3
votes
1answer
41 views

How to control for severe medical cases using survival analysis and Cox regression?

I have a longitudinal medical record dataset. My cohort is made up of patients with a particular disease. There are no members of this cohort without this disease. Disease indications are denoted by a ...
3
votes
1answer
188 views

Asymptotic variance of GMM with efficient instrument

This question emerged from reading Wooldridge's Econometric Analysis of Cross Section and Panel Data, second edition, section 14.4.3, where the asymptotic distribution of the GMM (Generalized Method ...
3
votes
1answer
367 views

Why does asymptotic efficiency require normality?

Greene (Econometric Analysis), states the defintiion of Asymptotic Efficiency. My question is, why does this definition contain a reference to normal distribution? We already know by the central ...
3
votes
1answer
458 views

Efficient estimator for the mean of a Gamma distribution

Let $X_1$,$X_2$,...,$X_n$ be i.i.d. according to Gamma($\alpha$,$\beta$). Denote the mean by $\mu := E[X_i] = \alpha/\beta$. Can you find an unbiased and efficient estimator for $\mu$? MLE gives ...
3
votes
1answer
772 views

Relative efficiency: mean deviation vs standard deviation

I have difficulties following a seemingly elementary claim from Tukey (1960): It is well known that, in large samples, the relative efficiency as a measure of scale of the mean deviation compared ...
3
votes
0answers
380 views

alternative to IQR [duplicate]

I am currently making a short literature study of robust and efficient estimators. Some very well known are the median absolute deviation (MAD) and the interquartile range (IQR). However they both ...
3
votes
0answers
75 views

Is the efficiency of biased estimators judged according to the the mean squared error criterion of optimality?

I understand that the Cramér–Rao bound relates to achieving the lowest possible mean squared error amongst unbiased estimators. Is the same standard used to judge biased estimators? Why/why not?
3
votes
0answers
243 views

A little confusion on Profile likelihood

As we all know, profile likelihood is an effective method for the estimation of conditional parametric model. But I still don't know exactly why it works. Profile likelihood was thoroughly studied by ...
2
votes
1answer
188 views

Distribution of $\bar{X^2} $ when $X\sim N \left( \theta, \sigma^2 \right) $

How can I derive the distribution of $$\bar{X^2}\quad \text{when}\quad X\sim N \left( \theta, \sigma^2 \right) $$ The context of this question is an exercise requiring me to show that $\bar{X^2}- \...
2
votes
2answers
47 views

How can I find the asymptotic relative efficiency of two quantities, estimating $\sigma$?

Let $X_1,...,X_n$ be a random sample from $N(0,\sigma^2)$, where $\sigma>0$ is unknown. We try to estimate $\sigma$ using $T_1=\sqrt{\frac{\pi}{2}}\frac{1}{n}\sum^n_{i=1}|X_i|$ and $T_2=\sqrt{\frac{...
2
votes
1answer
164 views

Using parallel tempering, is it possible to swap too often?

In parallel tempering I have replicas of the Markov chain I'm studying evolving at different temperatures, and intermittently I swap the replicas on a nearest-neighbouring temperatures basis. Between ...