Skip to main content

Questions tagged [efficiency]

A measure of the quality of a statistical estimator.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Maximum likelihood estimator when the Fisher information matrix is not invertible?

It is known that maximum likelihood estimator (MLE) is asymptotically efficient, i.e., $\sqrt{n\,}\,\left( \widehat{\theta \,}_{\mathrm{mle}}-\theta _0 \right) $ converges in distribution to a ...
narip's user avatar
  • 187
2 votes
0 answers
46 views

No Existence of Efficient estimator

I need to prove that given $(X_1,...,X_n)$ from the density $$\frac{1}{\theta}x^{\frac{1}{\theta}-1}1_{(0,1)}$$ no efficient estimator exists for $g(\theta)$=$\frac{1}{{\theta}+1}$. I have shown that ...
Onofrio Olivieri's user avatar
1 vote
0 answers
35 views

Small sample MLE vs OLS efficiency

MLE estimates are asymptotically efficient. Both MLE and OLS estimates are asymptotically normal and for many distributions their limiting variances coincide (information for one observation being the ...
memeplex's user avatar
  • 141
0 votes
0 answers
22 views

Is Coefficient of Variation a valid measure of relative efficiency?

I'm wondering if it is always valid to use Coefficient of Variation (CV) to determine relative efficiency of parameter estimators, and to compute statistically equivalent sample sizes based on that ...
feetwet's user avatar
  • 1,162
0 votes
0 answers
18 views

What is the difference between unbiasedness, consistency and efficiency of estimators? How are these interrelated among themselves? [duplicate]

!Efficiency(https://stackoverflow.com/20240427_193105.jpg). Given snapshot of the book states that among the class of consistent estimators, in general, more than one consistent estimator of a ...
Parth's user avatar
  • 1
0 votes
0 answers
23 views

What is known about the (in)efficiency of true score variance estimation?

In classical test theory, an observed score $X$ is defined as $$ X = T + E $$ where $T$ represents the individual true score and $E$ represents the individual error score. In this context $T$ may be ...
LJ Beinhauer's user avatar
0 votes
0 answers
15 views

Understanding Asymptotic Relative Efficiency and how to compute it

I am learning about asymptotic relative efficiency (ARE) in class, and I am trying to understand exactly how to compute the ARE. From my understanding, asymptotic relative efficiency refers to ...
Harry Lofi's user avatar
5 votes
2 answers
128 views

Sufficient conditions for asymptotic efficiency of MLE

Maximum-likelihood estimators are, according to Wikipedia, asymptotically efficient, that is they achieve the Cramér-Rao bound when sample size tends to infinity. But this seems to require some ...
Luis Mendo's user avatar
  • 1,099
5 votes
0 answers
27 views

Are these two estimated regression coefficient asymptotically equivalent? If not, which one is more efficient?

Suppose I have $Y=\beta_1X_1+\beta_2X_1X_2+g(X_2)+u$, where $E(u|X_1,X_2)=0$ and $S=g(X_2)+e$ with $E(e|X_2)=0$. I have a random sample $\{Y_i,X_{1i},X_{2i},S_i\}_{i=1}^n$. Suppose I first use a ...
ExcitedSnail's user avatar
  • 2,966
1 vote
1 answer
36 views

How can I rigorously quantify the increase performance due to additional parameters?

I am trying to evaluate a novel dimensionality reduction technique. Specifically, we start with a data set with around 1,000 features/covariates per observation. My technique maps this down to 12. ...
golfWolf's user avatar
2 votes
0 answers
78 views

How to show mathematically that random effects are more efficient than fixed effects?

I have read in several places now that random effects estimators are more efficient than fixed effects estimators, in particular here I’ve searched this site and Google and couldn’t find this result. ...
Geoff's user avatar
  • 741
3 votes
1 answer
144 views

Estimating ratio of regression coefficients

What is the best method of estimating a ratio of regression coefficients $\beta_1/\beta_2$ under the usual assumptions / in practice? I have two relatively well approximated signals $X_1, X_2$ and ...
Magemathician's user avatar
0 votes
0 answers
11 views

Questions about efficiency of estimators with longitudinal data

I'm modelling data with repeated observations; I'm reading up on options and pitfalls, and have a few questions. Coefficient estimates are still unbiased and consistent in the presence of ...
Geoff's user avatar
  • 741
6 votes
1 answer
372 views

Asymptotic efficiency of IQR

I was wondering about the asymptotic efficiency of the Interquartile Range (IQR) in the Gaussian case. I have calculated it empirically using a Monte Carlo estimator, and it appears to be equal to ...
zantoox's user avatar
  • 61
4 votes
1 answer
72 views

Is the smooth transformation of an asymptotically efficient estimator still asymptotically efficient?

Suppose I have an estimator $\widehat{\theta}$ that is asymptotically efficient (for example, it could be the MLE of the mean $\mu$ and variance $\sigma^2$ of $Normal(\mu,\sigma^2)$). Suppose I also ...
ExcitedSnail's user avatar
  • 2,966
3 votes
0 answers
39 views

Adjusting confidence interval of estimator by efficiency

Summary: If we have an unbiased MLE $\widehat{\sigma_1}$ of an exponential distribution parameter, and the confidence intervals for its estimates are given by the $\chi^2$ distribution; and we find ...
feetwet's user avatar
  • 1,162
0 votes
0 answers
23 views

Compare ARE of mean and median

I have followed some lectures on the internet on asymptotic relative efficiency. When I had calculated the relative efficiency $ARE(median, mean)$ than my result was $ARE(median, mean)=\frac{2}{\pi}\...
user210804's user avatar
2 votes
1 answer
133 views

Counterexamples: Minimax but not efficient, efficient but not minimax

Are there examples of estimators that are minimax, but not efficient? Perhaps, to be more concrete, any of the following: (strong) Estimator sequence for each $n$ that is minimax but does not match ...
JohnA's user avatar
  • 722
2 votes
1 answer
125 views

Derivation of the formula for the asymptotic relative efficiency of two estimators with different estimands

Background In their book, Huber & Ronchetti (pp. 2-3) compare the efficiency of the mean absolute deviation $d_n$ with the standard deviation $s_n$ with the following formula: $$ \operatorname{ARE}...
COOLSerdash's user avatar
0 votes
1 answer
88 views

How does one test the efficiency and completeness of an estimator using monte-carlo simulation?

How does one test the efficiency and completeness of an estimator using monte-carlo simulation? In particular, I want to use-montecarlo simuation to answer. Maybe the better question is how does one ...
user avatar
0 votes
0 answers
65 views

Efficiency of sample mean

Since the variance of the sample mean estimator decreases with $n$ $$ Var(\bar{X}) = \frac{\sigma^2}{n} $$ does that mean that it becomes more efficient as the sample size grows?
amrods's user avatar
  • 249
2 votes
1 answer
67 views

Comparing efficiency between estimators

Suppose that $\hat \theta_1, \hat \theta_2$ are two estimators of $\theta$. Furthermore, assume that \begin{align} \sqrt{n}(\hat \theta_1-\theta)\overset{d}{\to}N(0,V_1)\\ \sqrt{n}(\hat \theta_2-\...
Celine Harumi's user avatar
4 votes
2 answers
3k views

What does asymptotic efficiency mean?

I read some comparison articles, and always find "asymptotic efficiency," "asymptotically less efficient," and "asymptotically normal." I am really confused about the ...
Alice's user avatar
  • 650
2 votes
1 answer
330 views

Monte Carlo in R simulation for Efficiency [closed]

An exercise displayed in the image below shows example of finding the efficiency of an estimator. I am trying to replicate this example in R using monte carlo. Y1,Y2,Y3 are random samples of normal ...
confusedaboutstats's user avatar
1 vote
0 answers
83 views

unbiased estimator and efficiency

can someone plese clarify a doubt for me? Let (X1, . . . , Xn) be a random sample of i.i.d. random variables with expected value $µ$ and variance $σ^2$ Consider the following estimator of $µ$: $T_{n}(...
WNZ's user avatar
  • 21
1 vote
0 answers
75 views

Proving efficiency of GLS over OLS [duplicate]

I'm trying to prove that GLS is more efficient than OLS. I found out it is, I need to show that: $\text{Var}^{-1}({\beta}_{GLS})-\text{Var}^{-1}(\beta_{OLS})$ is the positive semi-definite matrix, ...
Qi Yao's user avatar
  • 19
2 votes
0 answers
611 views

Variable batch size for inputs of different length

We're fine-tuning a GPT-2 model (using the Adam optimizer) to some posts from a social network. The length of these posts varies quite dramatically, so while some are only two tokens long, others can ...
Christian Adam's user avatar
1 vote
0 answers
218 views

Does the null space of the design matrix X have an interpretation in statistics?

I am going through the material I learned in my linear algebra classes and I am trying to view the material from a statistical perspective. This typically boils down to imagining what the theorems say ...
confused student's user avatar
0 votes
2 answers
219 views

Find asymptotic efficiency of MLE to UMVUE

Let $\{X_i\}_{i=1}^n $ be a sequence of i.i.d random variables with common pdf: $$ f(x;a,\theta) =\theta a^\theta x^{-(\theta+1)} \boldsymbol 1_{(a,\infty)}(x) \, \,\text{; where } \theta, a > 0$$ ...
Short and Fuzzy's user avatar
2 votes
1 answer
557 views

Efficiency of IV vs GMM

I am trying to understand how IV/just identified GMM and overidentified GMM compare when it comes to efficiency. The way I understand it, we are able to identify the vector of coefficients in IV and ...
Blo4d's user avatar
  • 41
1 vote
1 answer
230 views

Is the Hodges-Lehmann estimator 'optimal' for estimating the location parameter of Logistic distribution?

Is the Hodges-Lehmann estimator $\hat\theta_{HL}=\operatorname{median}\limits_{1\le i\le j\le n}\left\{\frac{X_i+X_j}{2}\right\}$ in some sense 'optimal' for estimating the location parameter $\theta$ ...
StubbornAtom's user avatar
  • 11.5k
2 votes
1 answer
128 views

Stratified sampling ratio with srswor

The variable X under study have rectangular distribution with interval (a, a+ d ) the interval is divided into k equal subintervals which form k equal strata of equals sizes . From each stratum ...
simran's user avatar
  • 377
2 votes
0 answers
80 views

Is sample mean an 'efficient' estimator for alpha stable distribution?

The question is just like the title. But...$\alpha$-stable distribution (for $\alpha\in (1,2)$) does not have the second moment, so the sample mean doesn't have variance well defined. Then for such a ...
user avatar
0 votes
0 answers
682 views

Relative Efficiency of Sample Variance vs Square of Difference

$\newcommand{\szdp}[1]{\left(#1\right)} \newcommand{\szdb}[1]{\left[#1\right]} \newcommand{\eff}{\operatorname{eff}}$ Problem Statement: Suppose that $Y_1, Y_2,\dots, Y_n$ is a random sample from a ...
Adrian Keister's user avatar
2 votes
1 answer
242 views

Is BIC asymptotically efficient for minimizing prediction error if the true model is being considered?

If a set of models is being compared using BIC and AIC, given the fact that the true model (the one which generated the data) is in this set (and given the other assumptions that guarantee BIC ...
Marco Rudelli's user avatar
22 votes
11 answers
5k views

Does machine learning really need data-efficient algorithms?

Deep learning methods are often said to be very data-inefficient, requiring 100-1000 examples per class, where a human needs 1-2 to reach comparable classification accuracy. However, modern datasets ...
MWB's user avatar
  • 1,337
1 vote
1 answer
463 views

Solved: Relative Efficiency of Average versus Maximum Order Statistic on a Uniform Distribution

$\newcommand{\szdp}[1]{\!\left(#1\right)}\newcommand{\eff}{\operatorname{eff}}$ Problem Statement: Let $Y_1, Y_2, \dots, Y_n$ denote a random sample from the uniform distribution on the interval $(\...
Adrian Keister's user avatar
1 vote
0 answers
166 views

Asymptotic efficiency of estimators of autoregressive models

Are OLS or MLE estimators of autoregressive model asymptotically efficient if errors are i.i.d? Consider the case of an AR(1) model $$x_t=\alpha x_{t-1} + \epsilon_t$$ with $\epsilon_t$ ~ $i.i.d. N(0,\...
Oragonof's user avatar
4 votes
2 answers
410 views

Data-efficiency vs sample-efficiency

I noticed that in the context of RL, people call the ability to learn from little data "sample-efficiency". However, in the context of supervised learning, it is called "data-efficiency&...
MWB's user avatar
  • 1,337
0 votes
0 answers
43 views

An efficient estimator can be biased? [duplicate]

Can someone explain to me whether an efficient estimator can be biased? What I've read so far seems to be always about efficiency being a property being alluded to unbiased estimators but I was ...
Kurapika's user avatar
5 votes
3 answers
324 views

Efficient influence function in proportional hazards model

I was hoping someone could help me with this problem in the cox proportional hazards model. I am given the following setup. T is a non-negative random variable with continous distribution and hazard ...
nalen's user avatar
  • 63
0 votes
0 answers
49 views

How can I find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\theta}_{n}^{(2)}}$ (in terms of $\tau_1$, $\tau_2$ and $\alpha$)?

Given that $n^{\alpha}[\hat{\theta}_{n}^{(1)}−\theta_0] \xrightarrow{L} \tau_1 H$ and $n^{\alpha}[\hat{\theta}_{n}^{(2)}−\theta_0] \xrightarrow{L} \tau_2 H$, find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\...
anonyx2's user avatar
  • 33
2 votes
0 answers
34 views

Are these statements about the maximum likelihood estimator and efficiency correct?

I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced : 1/ If the maximum likelihood estimator ...
Hijaw's user avatar
  • 155
1 vote
1 answer
124 views

Expected variance of biased estimators

Suppose that $\mu$ is an unknown $k$-vector, which one seeks to guess. You observe $n$ i.i.d. normal variates $x_i \sim \mathcal{N}\left(\mu,\Sigma\right)$, say stacked in the matrix $X$, then produce ...
shabbychef's user avatar
  • 14.9k
3 votes
0 answers
61 views

Using calculus of variations to deduce lower bound on Pitman efficiency (asymptotic relative efficiency) between $t$-test and sign test

Let $Y_1,...,Y_n$ be iid draws from a location family $\{f(\cdot - \theta) : \theta \in \mathbb{R}\}$. $f$ is a symmetric density w.r.t. the Lebesgue measure on $\mathbb{R}$ with finite variance. We ...
martingale_50's user avatar
1 vote
1 answer
202 views

Accuracy of MLE

Hi I have a question related to MLE. Consider a simple linear regression model: y = a + b*x + u Here, if the regressor X has small variability, the OLE estimator could be inaccurate. My question is ...
M.C. Park's user avatar
  • 935
7 votes
0 answers
285 views

Why does MLE tend to normal distribution

We have $X_1,\dots, X_n$ are iid (the distribution can be of any type, e.g. Bernoulli (p), normal ($\mu, \sigma^2$), Poisson ($\lambda$). If we use MLE $\hat \theta$ to estimate any parameter $\theta$ ...
Charlie Chang's user avatar
2 votes
0 answers
2k views

MLE and consistency and efficiency of MLE

Here is my understanding of Likelihood function, maximum likelihood estimator (MLE) and consistency and efficiency of MLE. (Notes: Comments are not main parts of this post and can be skipped. Only ...
Charlie Chang's user avatar
4 votes
2 answers
111 views

Vector-valued estimators, intuitively why $var(\widetilde{\beta})-var(\widehat{\beta})$ being p.s.d. means $\widehat{\beta}$ more efficient?

For two scalar unbiased estimators $\widehat{\alpha}$ and $\widetilde{\alpha}$, we know that if one has smaller variance, then we say it is more efficient, which intuitively means that this estimator ...
ExcitedSnail's user avatar
  • 2,966
1 vote
0 answers
17 views

In regression modeling, are there any caveats to always using robust standard errors?

Aside from efficiency issues, is there anything else to this?
user321627's user avatar
  • 4,438