Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
2 votes
1 answer
25 views

Attainablility of Cramer Rao Bound with function of multi-parameters?

Suppose we have multivariables ${\boldsymbol {\theta }}=\left[\theta _{1},\theta _{2},\dots ,\theta _{d}\right]^{T}\in {\mathbb {R}}^{d}$, and we want to estimate the function of parameters $\...
  • 91
-1 votes
0 answers
5 views

Given that some statistics are estimators, are test-statistics consistent, efficient, complete and unbiased, estimators?

Given that some statistics are estimators, are test-statistics consistent, efficient, complete and unbiased, estimators? Are sufficient statistics consistent, efficient, complete and unbiased, ...
0 votes
1 answer
23 views

Derivation of the formula for the asymptotic relative efficiency of two estimators with different estimands

Background In their book, Huber & Ronchetti (pp. 2-3) compare the efficiency of the mean absolute deviation $d_n$ with the standard deviation $s_n$ with the following formula: $$ \operatorname{ARE}...
  • 26.4k
0 votes
0 answers
26 views

Showing the unbiased estimator of variance for GLS estimator

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
0 votes
0 answers
19 views

Find UMVUE of Ratio of two parametric functions

Let T be UMVUE of $g(\theta)$ and S be UMVUE of $h(\theta)$. Is there any way to find UMVUE of ratio of $g(\theta)$ and $h(\theta)$ i.e , $g(\theta)/h(\theta)$?
  • 1
0 votes
0 answers
26 views

Using consistency to prove unbiasedness of linear regression

I am not sure if this question has already been asked, but basically in an interview I was told that using consistency, we have that $$E(\frac{Cov(X,Y)}{Var(X)}) = \frac{E(Cov(X,Y))}{E(Var(X))}$$ ...
2 votes
1 answer
57 views

Is asymptotic unbiasedness different from unbiasedness in practice?

Given some estimator T for a parameter θ, by definition T is unbiased if its bias B(T) is 0. It is asymptotically unbiased if B(T) is not 0, but some value that tends to 0 as n goes to infinity. My ...
4 votes
1 answer
85 views

Unbiased estimator of $2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$

For a random variable $x$, how would I go about creating an unbiased estimator of the following quantity? $$R=2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$$ For instance, when $x$ comes from from chi-...
0 votes
1 answer
40 views

Confusion regarding proof that the variance estimator is unbiased for finite population

Going through Sharon L. Lohr's Sampling design book (2nd Edition), I have no issues with the content all the way until it goes into the proof in chapter 2 on SRSWOR that $E[s^2] = S^2$, where $S^2$ is ...
1 vote
0 answers
40 views

Linear regression on mean + standard deviation

I have observations made at different times from a normally distributed real random variable whose mean and standard deviation both vary linearly with time. How can I estimate these two linear ...
  • 111
1 vote
1 answer
51 views

why not using sample variance (instead of MSE) to estimate the error variance in linear regression?

Assuming the true equation for Y is linear as below: $$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$ Assuming X is fixed, then the variance of each Y is: $$var(Y_i )=var(\epsilon_i)=\sigma^2$$ In order to ...
  • 179
4 votes
1 answer
136 views

Unbiased estimator for $\mu_1/\mu_2$

Let $X_1,X_2,\ldots,X_n$ and $Y_1,Y_2,\ldots,Y_n$ be independent random samples from $N(\mu_1,1)$ and $N(\mu_2,1)$ populations respectively with $\mu_2\neq0$. I need to find an unbiased estimator for $...
  • 43
4 votes
0 answers
123 views

Unbiasing estimator of $\|\Sigma\|_F^2$

I have access to samples of some distribution with second-moment matrix $\Sigma=E[xx^T]$ and need an estimate of $\|\Sigma\|_F^2$ (which can be used to set optimal size for LMS) We can use Frobenius ...
0 votes
0 answers
20 views

Unbiasedness properties of Ratio/Proportion-type estimator

I have a ratio estimator, $\hat{a} = n_1/(n_0+n_1)$, where $n_x$ refers to the frequency of $x$-valued data. Note that, $E(n_0)$ and $E(n_1)$ exists and strictly positive. Usually, to show an ...
  • 312
4 votes
0 answers
35 views

How would you justify to a non-statistician why we should use an unbiased estimator instead of a maximum likelihood estimator?

Say we have the maximum likelihood estimator (which is usually biased) and an unbiased estimator and the sample size is small enough that these estimator are substantially different in magnitude. We'...
  • 1,173
1 vote
0 answers
24 views

Optimal combination of correlated estimations

Consider two random unbiased estimates $\hat X_1,$, $\hat X_2$ of a parameter (complex number) $x$, with estimation errors $E_1 = \hat X_1-x$, $E_2 = \hat X_2-x$. If the random variables $E_1$, $E_2$ ...
0 votes
0 answers
17 views

Find unbiased estimator for a and b based on X and Y

I need help with the following question(sorry for not formatting, I do not know how): X and Y are random variables, each have standard deviation of 3. The pearson correlation equals to 0.6(in this ...
  • 11
1 vote
1 answer
24 views

Is proportional hazard regression unbiased?

I am studying the results from a simple Cox PH regression, and I'm curious about the unbiasedness of the estimated log hazard ratios (i.e. coefficients) from the model: Are those estimates universally ...
1 vote
1 answer
23 views

Is the expected value of a correlation in a randomly permuted sample zero?

Let's say we have two real valued data sets $x$ and $y$, both of length $n$. I do not want to make any further assumptions regarding these data sets. We're interested in their correlation. For testing ...
0 votes
0 answers
34 views

Estimating subtotals from a simple random sample

Suppose, for illustration, that we have a population of $N=10$ enterprises $E_1,\cdots,E_{10}$. We extract a sample of $n=4$ enterprises by a simple random sampling method. The sampled enterprises ...
  • 115
0 votes
0 answers
57 views

Is collinearity really not a problem for GLM?

I have read that collinearity is not a problem for GLM. Is it really? I here estimate two models. The dependent variable is default, a dummy equals to 1 if someone ...
3 votes
0 answers
49 views

When can we get unbiased estimate given biased data?

There was a recent "hot take" tweet by Andrej Karpathy (without any comment or clarification from the author): real-world data distribution is ~N(0,1) good dataset is ~U(-2,2) It provoked ...
  • 121k
9 votes
1 answer
192 views

Finding the MVUE of the center of a circle of unknown location

Is there a known analytic solution for finding the minimum variance unbiased estimator of a disk of an unknown location given that a sample of $n$ points was drawn uniformly and randomly from the disk ...
  • 7,182
0 votes
0 answers
98 views

What is the difference between MVB UMVUE and MVUE.?

Cramer Rao inequality gives MVB and if MVB exist it is MLE. Rao Blackwell gives UMVUE, but isn’t when we have MVB estimator for unbiased it is UMVUE? Then what is MVUE? MVB minimum variance bound ...
2 votes
0 answers
48 views

What is the bias adjustment needed on k-folds CV? (made by cv.glm on R's boot package)

I'm trying to find the equation defined on the documentation of the cv.glm function in the boot package: "When $K$ is less ...
  • 21
2 votes
1 answer
106 views

Unbiased estimator of population variance for sampling without replacement

What I wrote below only apply to the situation where we have finite population. I saw many of my friends used sample variance with Bessel's correction $\frac{\sum_i^n (X_i - \bar{X})^2}{n-1}$ to ...
  • 259
2 votes
1 answer
62 views

Forecasting using regression coefficients

I have a regression-based model that is trained on market-level data that I'd like to use to make predictions on submarket level observations. For example, I fit the following model on market level ...
  • 488
2 votes
0 answers
44 views

What are some well-known unbiased estimator of regression coefficient besides OLS estimator?

Is there any other unbiased estimator of regression coefficient than OLS? For instance, one might consider using unbiased estimator with less computational cost (since OLS involves matrix inversion)?
0 votes
0 answers
37 views

Unbiased estimator of regression coefficient in high dimension

Is there any unbiased estimator for the regression coefficient $\beta \in \mathbb{R}^p$, p >> 1, where $$ y_k = x_k^T\beta + \epsilon \in \mathbb{R}? $$ Note that $x_k \in \mathbb{R}^p$ and $\...
1 vote
1 answer
40 views

Does an endogenous variable bias the coefficient of the exogenous one?

We have the following model: $$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon. $$ We know that: \begin{align*} \operatorname{Cov}(x_1, \epsilon) &\neq 0 \\ \operatorname{Cov}(x_2, \epsilon) &...
0 votes
0 answers
75 views

Weighted average estimator - unbiased and consistent

Take an estimator that produces a weighted average of all n observations in an i.i.d sample from a population with mean $\mu$ and variance $\sigma^2$. I.e.: $$ \bar{x}_w = \sum_{i=1}^{n} w_ix_i$$ ...
  • 13
0 votes
0 answers
19 views

Is the least mean square estimator for jointly gaussian variables necessarily affine?

In his book on adaptive filtering, Sayed mentions a subclass of affine estimators in which not only the predictions y are linearly dependent on the observations x, but x and y are jointly Gaussian. ...
1 vote
0 answers
76 views

Least square estimate expected value and variance of linear model

I am practice some exercises. Here it goes. "Assume we fit the simple model \begin{equation} \hskip 5cmy=X_1\beta_1+\epsilon \hskip 5cm (1) \end{equation} however the true model is \begin{...
3 votes
1 answer
72 views

Proof that $g(p)$ unbiasedly estimable only if it is a polynomial (Binomial Distribution)

In Lehmann-Casella (Theory of Point Estimation) they state without proof that if $T \sim Bin(n,p)$, then $g(p)$ is estimable only if it is a polynomial in $p$ of degree $\leq n$. How does one go about ...
0 votes
0 answers
13 views

Is the empirical distribution the only unbiased distribution estimator?

Given $n$ samples, if $\hat{p}$ is the empirical distribution of $p \in \Delta_{\mathcal{X}}$ where $\mathcal{X}$ is a finite domain, we know that $\mathbb{E} \hat{p} - p = 0$. Is the empirical ...
0 votes
0 answers
50 views

Proportion data: Logistic with MLE vs. OLS with logit-transformed response

This is an expansion of @Beethoven_90's comment on this question. Suppose I have proportion data $Y_i$ computed from a binomial; $Y_i = \frac{S_i}{N_i}$ where $S_i \sim Bin(N_i, p_i)$ and $p_i$ is the ...
  • 1
2 votes
0 answers
14 views

Calculating confidence interval for binomial distribution [duplicate]

Suppose we have a sample $X_1, X_2, \ldots, X_n \stackrel{\text{iid}}{\sim} Binomial(\theta)$, where $n$ is known to be large. I would like to calculate the 95% confidence interval for $\theta$, and I ...
  • 21
2 votes
0 answers
44 views

Expectation of Difference in Means estimator

Given i.i.d. observations $(Y_i, X_i)$ where $Y_i$ is the response and $X_i$ is binary valued, the difference in means estimator is $$ \hat{\theta} = \frac{1}{n_0} \sum_{i=1, X_i=0} Y_i - \frac{1}{n_1}...
  • 1,176
7 votes
1 answer
227 views

Find UMVUE of difference of parameters of two exponential distribution random variables

Let $X_{1}, \dots, X_{n}$ be i.i.d. having the exponential distribution $Exp\left(0, \theta_{x}\right)$ with $\theta_{x}>0$, and $Y_{1}, \dots, Y_{n}$ be i.i.d. having the exponential distribution $...
  • 95
1 vote
0 answers
56 views

Why can't OLS estimates be used to obtain regression parameters when dealing with high dimensional data?

Suppose I have a data set consisting of $n$ observations: ${\displaystyle \left\{\mathbf {x} _{i},y_{i}\right\}_{i=1}^{n}}$. If I apply linear regression :${\displaystyle \mathbf {y} =\mathrm {X} {\...
0 votes
1 answer
163 views

Proof that multicollinearity doesn't produce biased estimators

I'm trying to prove that multicollinearity doesn't introduce bias into a multiple linear regression model, but my proof seems to indicate the opposite. If we represent the model as $$y = \hat \beta_0 +...
0 votes
1 answer
65 views

Will removing a regressor from a model reduce the variance of the remaining regressor

Let's say our full model is a mean centered: $$ y= B_0 + B_1(x_1-\bar x_1) + B_2(x_2-\bar x_2) + e$$ I know $B_0$ works out to be equal to $\bar{y}$, and so $SS_{Reg}(B_0) = 0$ My question is if we ...
1 vote
0 answers
27 views

Existence of unbiased estimator for any $f(X)$? [closed]

Suppose you are handed a function of a random variable $f(X)$, how would you construct/rule out the existence of an unbiased estimator for it? I've read through Halmos (1946) but the characterization ...
0 votes
0 answers
59 views

Quantifying the bias of a quantile estimator based on order statistics, and its relation to asymptotic unbiasedness

From what I understand, the quantile estimator based on order statistics is asymptotically unbiased (and follows a Normal distribution). I have been looking for a quantification of the non-asymptotic ...
  • 1
1 vote
0 answers
33 views

How do we select model for causal inference?

I am reading Rubin's Causal Inference Sec 7.5 in context of completely randomized experiment. It says performing linear regression will produce asymptotically unbiased estimate of causal effect, ...
  • 1,179
2 votes
1 answer
155 views

How to show the Hansen-Hurwitz estimator is unbiased?

Consider a population of size $N$ and draw i.i.d. a random sample $S=(i_1,\dots,i_n)$ of $\{1,...,N\}$ with replacement. We define the Hansen-Hurwitz estimator as $$ \hat{\tau}= \frac{1}{n}\sum_{j=1}^...
  • 147
0 votes
0 answers
7 views

Why when the number of data increase the consistency can’t guarantee that the bias induced by the estimator diminishes

Consistency ensures that the bias induced by the estimator decreases as the number of data examples increases. However, the converse is not true asymptotically, an unbiased estimator does not imply ...
0 votes
0 answers
15 views

Difference in function mean between two groups with noisy group membership

Suppose that I have a distribution $\mathbb P_X$ over a space $\mathcal X$, and each $x\in \mathcal X$ belongs to either group 0 or group 1, according to a (deterministic) membership function $m: \...
  • 225
1 vote
1 answer
49 views

Do robust estimators like M-estimator still have higher variance than OLS in presence of non-normal errors and/or outliers?

In my studies I've learned that even with non-normality of the errors, the OLS estimator is still considered BLUE (Best Linear Unbiased Estimator). The texts also suggested using M and L estimators ...
  • 161
1 vote
0 answers
40 views

unbiased estimator and efficiency

can someone plese clarify a doubt for me? Let (X1, . . . , Xn) be a random sample of i.i.d. random variables with expected value $µ$ and variance $σ^2$ Consider the following estimator of $µ$: $T_{n}(...
  • 11

1
2 3 4 5
15