Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
745
questions
2
votes
1
answer
94
views
Is asymptotic unbiasedness different from unbiasedness in practice?
Given some estimator T for a parameter θ, by definition T is unbiased if its bias B(T) is 0. It is asymptotically unbiased if B(T) is not 0, but some value that tends to 0 as n goes to infinity. My ...
4
votes
1
answer
93
views
Unbiased estimator of $2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$
For a random variable $x$, how would I go about creating an unbiased estimator of the following quantity?
$$R=2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$$
For instance, when $x$ comes from from chi-...
- 5,967
0
votes
1
answer
74
views
Confusion regarding proof that the variance estimator is unbiased for finite population
Going through Sharon L. Lohr's Sampling design book (2nd Edition), I have no issues with the content all the way until it goes into the proof in chapter 2 on SRSWOR that $E[s^2] = S^2$, where $S^2$ is ...
- 103
1
vote
0
answers
143
views
Linear regression on mean + standard deviation
I have observations made at different times from a normally distributed real random variable whose mean and standard deviation both vary linearly with time. How can I estimate these two linear ...
- 111
2
votes
1
answer
148
views
why not using sample variance (instead of MSE) to estimate the error variance in linear regression?
Assuming the true equation for Y is linear as below:
$$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$
Assuming X is fixed, then the variance of each Y is:
$$var(Y_i )=var(\epsilon_i)=\sigma^2$$
In order to ...
- 319
4
votes
1
answer
197
views
Unbiased estimator for $\mu_1/\mu_2$
Let $X_1,X_2,\ldots,X_n$ and $Y_1,Y_2,\ldots,Y_n$ be independent random samples from $N(\mu_1,1)$ and $N(\mu_2,1)$ populations respectively with $\mu_2\neq0$.
I need to find an unbiased estimator for $...
- 43
4
votes
0
answers
138
views
Unbiasing estimator of $\|\Sigma\|_F^2$
I have access to samples of some distribution with second-moment matrix $\Sigma=E[xx^T]$ and need an estimate of $\|\Sigma\|_F^2$ (which can be used to set optimal size for LMS)
We can use Frobenius ...
- 5,967
0
votes
0
answers
33
views
Unbiasedness properties of Ratio/Proportion-type estimator
I have a ratio estimator, $\hat{a} = n_1/(n_0+n_1)$, where $n_x$ refers to the frequency of $x$-valued data. Note that, $E(n_0)$ and $E(n_1)$ exists and strictly positive.
Usually, to show an ...
- 364
5
votes
0
answers
48
views
How would you justify to a non-statistician why we should use an unbiased estimator instead of a maximum likelihood estimator?
Say we have the maximum likelihood estimator (which is usually biased) and an unbiased estimator and the sample size is small enough that these estimator are substantially different in magnitude. We'...
- 1,233
1
vote
0
answers
31
views
Optimal combination of correlated estimations
Consider two random unbiased estimates $\hat X_1,$, $\hat X_2$ of a parameter (complex number) $x$, with estimation errors $E_1 = \hat X_1-x$, $E_2 = \hat X_2-x$.
If the random variables $E_1$, $E_2$ ...
- 885
0
votes
0
answers
24
views
Find unbiased estimator for a and b based on X and Y
I need help with the following question(sorry for not formatting, I do not know how):
X and Y are random variables, each have standard deviation of 3. The pearson correlation equals to 0.6(in this ...
- 11
1
vote
1
answer
49
views
Is proportional hazard regression unbiased?
I am studying the results from a simple Cox PH regression, and I'm curious about the unbiasedness of the estimated log hazard ratios (i.e. coefficients) from the model:
Are those estimates universally ...
- 85
1
vote
1
answer
40
views
Is the expected value of a correlation in a randomly permuted sample zero?
Let's say we have two real valued data sets $x$ and $y$, both of length $n$. I do not want to make any further assumptions regarding these data sets. We're interested in their correlation. For testing ...
- 17.8k
0
votes
0
answers
34
views
Estimating subtotals from a simple random sample
Suppose, for illustration, that we have a population of $N=10$ enterprises $E_1,\cdots,E_{10}$. We extract a sample of $n=4$ enterprises by a simple random sampling method. The sampled enterprises ...
- 115
3
votes
0
answers
77
views
When can we get unbiased estimate given biased data?
There was a recent "hot take" tweet by Andrej Karpathy (without any comment or clarification from the author):
real-world data distribution is ~N(0,1)
good dataset is ~U(-2,2)
It provoked ...
Tim♦
- 130k
9
votes
1
answer
207
views
Finding the MVUE of the center of a circle of unknown location
Is there a known analytic solution for finding the minimum variance unbiased estimator of a disk of an unknown location given that a sample of $n$ points was drawn uniformly and randomly from the disk ...
- 7,362
0
votes
0
answers
237
views
What is the difference between MVB UMVUE and MVUE.?
Cramer Rao inequality gives MVB and if MVB exist it is MLE.
Rao Blackwell gives UMVUE, but isn’t when we have MVB estimator for unbiased it is UMVUE?
Then what is MVUE?
MVB minimum variance bound
...
- 13
2
votes
0
answers
95
views
What is the bias adjustment needed on k-folds CV? (made by cv.glm on R's boot package)
I'm trying to find the equation defined on the documentation of the cv.glm function in the boot package: "When $K$ is less ...
- 31
2
votes
1
answer
407
views
Unbiased estimator of population variance for sampling without replacement
What I wrote below only apply to the situation where we have finite population.
I saw many of my friends used sample variance with Bessel's correction $\frac{\sum_i^n (X_i - \bar{X})^2}{n-1}$ to ...
user340483
3
votes
1
answer
117
views
Forecasting using regression coefficients
I have a regression-based model that is trained on market-level data that I'd like to use to make predictions on submarket level observations.
For example, I fit the following model on market level ...
- 528
2
votes
0
answers
57
views
What are some well-known unbiased estimator of regression coefficient besides OLS estimator?
Is there any other unbiased estimator of regression coefficient than OLS? For instance, one might consider using unbiased estimator with less computational cost (since OLS involves matrix inversion)?
- 21
0
votes
0
answers
53
views
Unbiased estimator of regression coefficient in high dimension
Is there any unbiased estimator for the regression coefficient $\beta \in \mathbb{R}^p$, p >> 1, where
$$
y_k = x_k^T\beta + \epsilon \in \mathbb{R}?
$$
Note that $x_k \in \mathbb{R}^p$ and $\...
- 21
2
votes
1
answer
91
views
Does an endogenous variable bias the coefficient of the exogenous one?
We have the following model:
$$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon. $$
We know that:
\begin{align*}
\operatorname{Cov}(x_1, \epsilon) &\neq 0 \\
\operatorname{Cov}(x_2, \epsilon) &...
- 1,928
0
votes
0
answers
19
views
Is the least mean square estimator for jointly gaussian variables necessarily affine?
In his book on adaptive filtering, Sayed mentions a subclass of affine estimators in which not only the predictions y are linearly dependent on the observations x, but x and y are jointly Gaussian. ...
1
vote
0
answers
123
views
Least square estimate expected value and variance of linear model
I am practice some exercises. Here it goes.
"Assume we fit the simple model \begin{equation}
\hskip 5cmy=X_1\beta_1+\epsilon \hskip 5cm (1)
\end{equation}
however the true model is \begin{...
- 21
3
votes
1
answer
163
views
Proof that $g(p)$ unbiasedly estimable only if it is a polynomial (Binomial Distribution)
In Lehmann-Casella (Theory of Point Estimation) they state without proof that if $T \sim Bin(n,p)$, then $g(p)$ is estimable only if it is a polynomial in $p$ of degree $\leq n$. How does one go about ...
- 61
2
votes
0
answers
19
views
Calculating confidence interval for binomial distribution [duplicate]
Suppose we have a sample $X_1, X_2, \ldots, X_n \stackrel{\text{iid}}{\sim} Binomial(\theta)$, where $n$ is known to be large. I would like to calculate the 95% confidence interval for $\theta$, and I ...
- 21
2
votes
0
answers
69
views
Expectation of Difference in Means estimator
Given i.i.d. observations $(Y_i, X_i)$ where $Y_i$ is the response and $X_i$ is binary valued, the difference in means estimator is
$$
\hat{\theta} = \frac{1}{n_0} \sum_{i=1, X_i=0} Y_i - \frac{1}{n_1}...
- 1,398
7
votes
1
answer
310
views
Find UMVUE of difference of parameters of two exponential distribution random variables
Let $X_{1}, \dots, X_{n}$ be i.i.d. having the exponential distribution $Exp\left(0, \theta_{x}\right)$ with $\theta_{x}>0$, and $Y_{1}, \dots, Y_{n}$ be i.i.d. having the exponential distribution $...
- 115
1
vote
0
answers
98
views
Why can't OLS estimates be used to obtain regression parameters when dealing with high dimensional data?
Suppose I have a data set consisting of $n$ observations: ${\displaystyle \left\{\mathbf {x} _{i},y_{i}\right\}_{i=1}^{n}}$.
If I apply linear regression :${\displaystyle \mathbf {y} =\mathrm {X} {\...
- 21
0
votes
1
answer
419
views
Proof that multicollinearity doesn't produce biased estimators
I'm trying to prove that multicollinearity doesn't introduce bias into a multiple linear regression model, but my proof seems to indicate the opposite.
If we represent the model as
$$y = \hat \beta_0 +...
- 21
0
votes
1
answer
147
views
Will removing a regressor from a model reduce the variance of the remaining regressor
Let's say our full model is a mean centered:
$$ y= B_0 + B_1(x_1-\bar x_1) + B_2(x_2-\bar x_2) + e$$
I know $B_0$ works out to be equal to $\bar{y}$, and so $SS_{Reg}(B_0) = 0$
My question is if we ...
- 21
1
vote
0
answers
47
views
Existence of unbiased estimator for any $f(X)$? [closed]
Suppose you are handed a function of a random variable $f(X)$, how would you construct/rule out the existence of an unbiased estimator for it?
I've read through Halmos (1946) but the characterization ...
- 61
1
vote
0
answers
96
views
Quantifying the bias of a quantile estimator based on order statistics, and its relation to asymptotic unbiasedness
From what I understand, the quantile estimator based on order statistics is asymptotically unbiased (and follows a Normal distribution).
I have been looking for a quantification of the non-asymptotic ...
- 11
1
vote
0
answers
41
views
How do we select model for causal inference?
I am reading Rubin's Causal Inference Sec 7.5 in context of completely randomized experiment.
It says performing linear regression will produce asymptotically unbiased estimate of causal effect, ...
- 1,335
2
votes
1
answer
407
views
How to show the Hansen-Hurwitz estimator is unbiased?
Consider a population of size $N$ and draw i.i.d. a random sample $S=(i_1,\dots,i_n)$ of $\{1,...,N\}$ with replacement. We define the Hansen-Hurwitz estimator as
$$
\hat{\tau}= \frac{1}{n}\sum_{j=1}^...
- 147
1
vote
1
answer
127
views
Do robust estimators like M-estimator still have higher variance than OLS in presence of non-normal errors and/or outliers?
In my studies I've learned that even with non-normality of the errors, the OLS estimator is still considered BLUE (Best Linear Unbiased Estimator). The texts also suggested using M and L estimators ...
- 321
1
vote
0
answers
73
views
unbiased estimator and efficiency
can someone plese clarify a doubt for me?
Let (X1, . . . , Xn) be a random sample of i.i.d. random variables with expected value $µ$ and variance $σ^2$
Consider the following estimator of $µ$:
$T_{n}(...
- 11
0
votes
0
answers
56
views
what is the probability of sample variance when true variance and true mean is unknown?
Sample Variance by definition is $s^2 =\frac{1}{n-1} \sum{(x_i-\bar{x})^2}$
When the population distribution is normal and true variance $\sigma^2$ is known, Sample Variance follows the chisq ...
- 123
1
vote
1
answer
45
views
Can you correct "bias" in a regression if you can measure/model it? A journey in missing data and reweighting test scores
Thank you for joining me on this semi-theoretical journey. Here we will discuss how to account for "predictable" bias in your data.
Let's say we have a test composed on many subtests. A ...
- 183
0
votes
0
answers
44
views
what is the bias and variance of this LS estimator?
I want to estimate the variables $a$ and $b$ ($\theta = \left[ {\begin{array}{*{20}{c}}
a\\
b
\end{array}} \right]$) in the nonlinear model:
$$y\left( t \right) = au\left( t \right) + b\exp (u(t)) + ...
2
votes
0
answers
103
views
Symmetric distribution with defined mean: is the median always unbiased for the mean?
Let $X_1,\cdots,X_n\overset{iid}{\sim} F_X(x)$ be a random sample from a symmetric distribution with a defined mean. If need be, assume that $n$ is odd and that $F_X(x)$ is continuous.
Is it always ...
- 50.7k
0
votes
1
answer
49
views
Unbiased least squares estimate for GM Theorem
In order to prove the Gauss-Markov Theorem, we first have to show that the OLS estimate $\hat{\theta}$ is an unbiased estimator. From what Im reading on Internet and some textbooks, these are the main ...
- 123
4
votes
0
answers
261
views
In Ordinary Least Square (OLS) estimation: is the slope actually an "Inverse-variance weighting" estimator?
I am suspecting the answer is yes, but I'd appreciate help in proving it (even though we know that the estimator is BLUE, so it should probably hold).
For context:
An Inverse-variance weighting is ...
- 20.8k
1
vote
0
answers
64
views
How to find an unbiased estimator for reciprocal of scale parameter given an iid exponential sample?
For a random sample $X_1, ..., X_n$ from an exponential distribution with scale parameter $\lambda$, the density is given by $f(x) = \frac{1}{\lambda}e^{-\frac{1}{\lambda}x}; \,x \geq 0,\, \lambda >...
- 594
1
vote
1
answer
195
views
Show unbiased OLS estimator and expression for variance of OLS estimator
Consider the usual linear mixed model:
$$Y=X \beta+ZB+\epsilon $$
where Y and $\epsilon$ are $n$-dimensional random variables and $B$ is a $q$-dimensional random variable independent of $\epsilon$ so ...
- 315
0
votes
0
answers
21
views
Correct bias with known DAG
I have the following causal graph:
$T \to P$
$(T, P) \to S$
So $T$ causes $P$ (partially) and $T$ and $P$ both cause $S.$ If I just regress $S\,\text{~}\,T + P,$ I will get an overestimated effect for ...
- 263
0
votes
1
answer
101
views
Unbiased Estimator for Mean Response to Treatment
$\newcommand{\eps}{\varepsilon}\newcommand{\szdp}[1]{\!\left(#1\right)}$
Problem Statement: Consider the following model for the responses measured in a randomized block design containing $b$ blocks ...
- 5,605
1
vote
0
answers
47
views
Deriving unbiasedness of estimators (involving method of moments idea) of normal distribution with heterogeneous variance
I am currently reading this paper and in pp.127, 128, there are unbiased estimators that I cannot derive its unbiasedness.
The setting is simple. Let $$X_i\sim N(\mu,\tau^2+\sigma_i^2),\quad i\in\{1,\...
- 33
1
vote
0
answers
70
views
Unbiased estimator of $1/(1-a)$
Let $X_1,...,X_n$ be Poisson with parameter $a$. I am looking for a unbiased estimator of $h(a)=\frac{1}{1-a}$
Let $T$ be a statistic and $g(t)$ be it's pmf. Then if we have $E(T)= h(a)$ then $T$ is ...
- 123