Questions tagged [efficiency]

A measure of the quality of a statistical estimator.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Calculating "relative efficiency" of a single estimator in two samples from different processes

I want to assess how well the multilevel AR(1) model performs on not-so-long time series wherein the AR(1) model assumptions are violated. I have three data generating processes (say, $P_1$, $P_2$, ...
user avatar
  • 159
2 votes
1 answer
81 views

Monte Carlo in R simulation for Efficiency [closed]

An exercise displayed in the image below shows example of finding the efficiency of an estimator. I am trying to replicate this example in R using monte carlo. Y1,Y2,Y3 are random samples of normal ...
user avatar
0 votes
0 answers
23 views

Relative efficiency of two estimators for the cardinality of a set from which we sample without replacement

I would like to check my understanding of the relative efficiency for the following estimators for the problem of estimating the cardinality of a set, from which you sample without replacement; i.e. ...
user avatar
  • 115
1 vote
0 answers
30 views

unbiased estimator and efficiency

can someone plese clarify a doubt for me? Let (X1, . . . , Xn) be a random sample of i.i.d. random variables with expected value $µ$ and variance $σ^2$ Consider the following estimator of $µ$: $T_{n}(...
user avatar
  • 11
1 vote
0 answers
40 views

Proving efficiency of GLS over OLS [duplicate]

I'm trying to prove that GLS is more efficient than OLS. I found out it is, I need to show that: $\text{Var}^{-1}({\beta}_{GLS})-\text{Var}^{-1}(\beta_{OLS})$ is the positive semi-definite matrix, ...
user avatar
  • 19
1 vote
0 answers
73 views

Variable batch size for inputs of different length

We're fine-tuning a GPT-2 model (using the Adam optimizer) to some posts from a social network. The length of these posts varies quite dramatically, so while some are only two tokens long, others can ...
user avatar
0 votes
0 answers
36 views

Does the null space of the design matrix X have an interpretation in statistics?

I am going through the material I learned in my linear algebra classes and I am trying to view the material from a statistical perspective. This typically boils down to imagining what the theorems say ...
user avatar
0 votes
0 answers
26 views

Simulations: why is "1 by 1" much more efficient than "many by 1"?

Note: I am not familiar with discussions of this particular issue of simulation studies, so I may use wrong terms or oversee obvious aspect. My apologies for that. I want to simulate a two-step ...
user avatar
  • 1,552
0 votes
2 answers
82 views

Find asymptotic efficiency of MLE to UMVUE

Let $\{X_i\}_{i=1}^n $ be a sequence of i.i.d random variables with common pdf: $$ f(x;a,\theta) =\theta a^\theta x^{-(\theta+1)} \boldsymbol 1_{(a,\infty)}(x) \, \,\text{; where } \theta, a > 0$$ ...
user avatar
2 votes
1 answer
131 views

Efficiency of IV vs GMM

I am trying to understand how IV/just identified GMM and overidentified GMM compare when it comes to efficiency. The way I understand it, we are able to identify the vector of coefficients in IV and ...
user avatar
  • 31
1 vote
1 answer
70 views

Is the Hodges-Lehmann estimator 'optimal' for estimating the location parameter of Logistic distribution?

Is the Hodges-Lehmann estimator $\hat\theta_{HL}=\operatorname{median}\limits_{1\le i\le j\le n}\left\{\frac{X_i+X_j}{2}\right\}$ in some sense 'optimal' for estimating the location parameter $\theta$ ...
user avatar
  • 9,120
2 votes
1 answer
45 views

Stratified sampling ratio with srswor

The variable X under study have rectangular distribution with interval (a, a+ d ) the interval is divided into k equal subintervals which form k equal strata of equals sizes . From each stratum ...
user avatar
  • 437
2 votes
0 answers
22 views

Is sample mean an 'efficient' estimator for alpha stable distribution?

The question is just like the title. But...$\alpha$-stable distribution (for $\alpha\in (1,2)$) does not have the second moment, so the sample mean doesn't have variance well defined. Then for such a ...
user avatar
0 votes
0 answers
82 views

Relative Efficiency of Sample Variance vs Square of Difference

$\newcommand{\szdp}[1]{\left(#1\right)} \newcommand{\szdb}[1]{\left[#1\right]} \newcommand{\eff}{\operatorname{eff}}$ Problem Statement: Suppose that $Y_1, Y_2,\dots, Y_n$ is a random sample from a ...
user avatar
2 votes
1 answer
98 views

Is BIC asymptotically efficient for minimizing prediction error if the true model is being considered?

If a set of models is being compared using BIC and AIC, given the fact that the true model (the one which generated the data) is in this set (and given the other assumptions that guarantee BIC ...
user avatar
22 votes
11 answers
5k views

Does machine learning really need data-efficient algorithms?

Deep learning methods are often said to be very data-inefficient, requiring 100-1000 examples per class, where a human needs 1-2 to reach comparable classification accuracy. However, modern datasets ...
user avatar
  • 1,173
0 votes
0 answers
10 views

What is the relationship between "convergence rate" and "sample efficiency"?

It seems that saying "the algo would converge slowly" and saying "the algo would have a low sample efficiency" mean something similar. Do the two concepts describe different facets ...
user avatar
  • 796
1 vote
1 answer
88 views

Solved: Relative Efficiency of Average versus Maximum Order Statistic on a Uniform Distribution

$\newcommand{\szdp}[1]{\!\left(#1\right)}\newcommand{\eff}{\operatorname{eff}}$ Problem Statement: Let $Y_1, Y_2, \dots, Y_n$ denote a random sample from the uniform distribution on the interval $(\...
user avatar
0 votes
0 answers
14 views

fixed effects model: efficiency

Suppose I have flight delay data for multiple airlines. I want to estimate the fixed-effects model and choose aircraft to be a cross-sectional unit. Therefore, I include aircraft fixed effects in the ...
user avatar
  • 27
1 vote
0 answers
73 views

Asymptotic efficiency of estimators of autoregressive models

Are OLS or MLE estimators of autoregressive model asymptotically efficient if errors are i.i.d? Consider the case of an AR(1) model $$x_t=\alpha x_{t-1} + \epsilon_t$$ with $\epsilon_t$ ~ $i.i.d. N(0,\...
user avatar
3 votes
2 answers
71 views

Data-efficiency vs sample-efficiency

I noticed that in the context of RL, people call the ability to learn from little data "sample-efficiency". However, in the context of supervised learning, it is called "data-efficiency&...
user avatar
  • 1,173
0 votes
0 answers
38 views

An efficient estimator can be biased? [duplicate]

Can someone explain to me whether an efficient estimator can be biased? What I've read so far seems to be always about efficiency being a property being alluded to unbiased estimators but I was ...
user avatar
5 votes
2 answers
136 views

Efficient influence function in proportional hazards model

I was hoping someone could help me with this problem in the cox proportional hazards model. I am given the following setup. T is a non-negative random variable with continous distribution and hazard ...
user avatar
  • 63
0 votes
0 answers
46 views

How can I find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\theta}_{n}^{(2)}}$ (in terms of $\tau_1$, $\tau_2$ and $\alpha$)?

Given that $n^{\alpha}[\hat{\theta}_{n}^{(1)}−\theta_0] \xrightarrow{L} \tau_1 H$ and $n^{\alpha}[\hat{\theta}_{n}^{(2)}−\theta_0] \xrightarrow{L} \tau_2 H$, find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\...
user avatar
  • 23
2 votes
0 answers
31 views

Are these statements about the maximum likelihood estimator and efficiency correct?

I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced : 1/ If the maximum likelihood estimator ...
user avatar
  • 155
0 votes
1 answer
83 views

Expected variance of biased estimators

Suppose that $\mu$ is an unknown $k$-vector, which one seeks to guess. You observe $n$ i.i.d. normal variates $x_i \sim \mathcal{N}\left(\mu,\Sigma\right)$, say stacked in the matrix $X$, then produce ...
user avatar
  • 10.4k
3 votes
0 answers
39 views

Using calculus of variations to deduce lower bound on Pitman efficiency (asymptotic relative efficiency) between $t$-test and sign test

Let $Y_1,...,Y_n$ be iid draws from a location family $\{f(\cdot - \theta) : \theta \in \mathbb{R}\}$. $f$ is a symmetric density w.r.t. the Lebesgue measure on $\mathbb{R}$ with finite variance. We ...
user avatar
1 vote
1 answer
60 views

Accuracy of MLE

Hi I have a question related to MLE. Consider a simple linear regression model: y = a + b*x + u Here, if the regressor X has small variability, the OLE estimator could be inaccurate. My question is ...
user avatar
  • 501
7 votes
0 answers
219 views

Why does MLE tend to normal distribution

We have $X_1,\dots, X_n$ are iid (the distribution can be of any type, e.g. Bernoulli (p), normal ($\mu, \sigma^2$), Poisson ($\lambda$). If we use MLE $\hat \theta$ to estimate any parameter $\theta$ ...
user avatar
1 vote
0 answers
527 views

MLE and consistency and efficiency of MLE

Here is my understanding of Likelihood function, maximum likelihood estimator (MLE) and consistency and efficiency of MLE. (Notes: Comments are not main parts of this post and can be skipped. Only ...
user avatar
4 votes
2 answers
71 views

Vector-valued estimators, intuitively why $var(\widetilde{\beta})-var(\widehat{\beta})$ being p.s.d. means $\widehat{\beta}$ more efficient?

For two scalar unbiased estimators $\widehat{\alpha}$ and $\widetilde{\alpha}$, we know that if one has smaller variance, then we say it is more efficient, which intuitively means that this estimator ...
user avatar
  • 1,682
1 vote
0 answers
14 views

In regression modeling, are there any caveats to always using robust standard errors?

Aside from efficiency issues, is there anything else to this?
user avatar
  • 2,609
2 votes
2 answers
507 views

How can I find the asymptotic relative efficiency of two quantities, estimating $\sigma$?

Let $X_1,...,X_n$ be a random sample from $N(0,\sigma^2)$, where $\sigma>0$ is unknown. We try to estimate $\sigma$ using $T_1=\sqrt{\frac{\pi}{2}}\frac{1}{n}\sum^n_{i=1}|X_i|$ and $T_2=\sqrt{\frac{...
user avatar
  • 1,595
1 vote
0 answers
94 views

Deriving C-R inequality from H-C-R bound

As mentioned in the title, I want to derive the Cramer-Rao Lower bound from the Hammersly-Chapman-Robbins lower bound for the variance of a statistic $T$. The statement for the H-C-R lower bound is ...
user avatar
1 vote
1 answer
295 views

Does a balanced design have to be connected?

Under a general block design setup, consider $v$ treatments allocated to $b$ blocks. Let $r_i$ and $k_j$ be the replication number of the $i$th treatment and $j$ the block respectively. We define $n_{...
user avatar
  • 9,120
4 votes
1 answer
103 views

Which is a better estimator, averaged functions vs. A function of an average?

Problem: Assume that we want to estimate $f(\theta)$ with a pre-specified strictly increasing function $f$ and a parameter $\theta$. Let $\hat{\theta}_1$ and $\hat{\theta}_2$ be unbiased estimators ...
user avatar
  • 529
2 votes
1 answer
184 views

Are GAM suitable for inference?

Are the estimators unbiased efficient and consistent? Or is GAM better for classification and prediction than non additive models? Interaction terms aren’t allowed in GAM.
user avatar
1 vote
0 answers
70 views

Is the $\sigma$ estimator more efficient than the $\mu$ estimator?

Are my empirical findings correct? How to get the same result analytically? I studied the efficiency of the mean and standard dev estimators: $$\mu_n=\sum \frac {x_i} {n}\space\space\space\space\...
user avatar
  • 1,048
0 votes
0 answers
159 views

Exponential family and efficient estimator

In my lecture notes there is the notion of efficiency related to the exponential family. More precisely, the lecturer stated that for an exponential family an efficient estimator always exists. How is ...
user avatar
3 votes
1 answer
337 views

In the case of lognormal distribution, when can the median be more efficiently estimated than the mean?

I understand, that in the case of normal distribution, the estimation of the mean (from samples) is more efficient (i.e. of less risk), than the estimation of the median. According e.g. to this post, ...
user avatar
2 votes
0 answers
78 views

Efficiency of OLS versus Quantile regression estimator

If I have a linear model $ y_i = x_i'\boldsymbol\beta + \epsilon_i $ and I assume that OLS estimator of $\boldsymbol\beta$ is unbiased and consistent and Least absolute deviation (LAD) estimator of $...
user avatar
  • 127
1 vote
0 answers
41 views

Quantile regression estimator - conditions for consistency and efficiency

What are the conditions for consistency and efficiency of Quantile regression estimator (for example LAD) in a linear regression model?
user avatar
  • 127
9 votes
1 answer
3k views

When can't Cramer-Rao lower bound be reached?

The Cramer-Rao lower bound (CRLB) gives the minimum variance of an unbiased estimator. One sentence in the wiki page says "However, in some cases, no unbiased technique exists which achieves the bound....
user avatar
  • 210
2 votes
0 answers
208 views

FGLS vs OLS tradeoff

Hi, Can someone tell me what is the tradeoff between using OLS and FGLS? I know the conditions under which FGLS is more efficient than OLS. But what about robustness? It is usually said that the ...
user avatar
3 votes
1 answer
647 views

What may be an inefficient estimator of the population mean?

If the sample mean is an efficient estimator of the population mean, what may be an example of an inefficient such estimator?
user avatar
4 votes
2 answers
181 views

ARMA Estimation Efficiency of the Mean

Usually in books related to ARMA Time Series, it is assumed that the series is 0 mean. If not, the recommended standard procedure is to simply subtract the sample mean from the series and continue ...
user avatar
5 votes
1 answer
2k views

What is the difference between BLUE and MVUE?

What is the difference between a Best Linear Unbiased Estimator (BLUE) and a Minimum Variance Unbiased Estimator (MVUE)? I know that "best" mean efficient, but that is also what "minimum variance" ...
user avatar
  • 207
4 votes
0 answers
258 views

Minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed

What is the minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed? median When we wish to estimate the median, $\mu$, of a normal distributed variable then ...
user avatar
2 votes
1 answer
2k views

QR decomposition computational efficiency

I am struggling to find a reference for this: In terms of big Oh notation does anyone know of any expressions for the computational time taken by commonly used algorithms for QR decompositions?
user avatar
  • 576
0 votes
1 answer
45 views

Speeding up in R [closed]

for(i in 1 : n){ y[i] <- length(X[X >= X[i]]) } This is my code in R (partially given) to obtain the number of X's greater than or equal to each Xi, where X is a vector of values. When I run ...
user avatar