Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

0
votes
0answers
11 views

When is the bias of a statistic of the form a/n + b/n^2 + c/n^3 +

In many books the bias-correction of the Jackknife resampling method is being prooved under the assumption, that the bias has a special form, namely a/n + b/n^2 + c/n^3 * ... Sometimes it's written "...
1
vote
0answers
26 views

Do unbiased regression coefficents yield better prediction?

I ask myself if a have a omitted variables bias in my regression modell the coefficients of the model are biased so the mse growth because this coefficents are biased right? So does it mean if i ...
0
votes
0answers
23 views

Find $k \in R$ such that $P\left(\max\left\{\frac{{S_x}^2}{{S_y}^2}, \frac{{S_y}^2}{{S_x}^2}\right\} > k\right)= 0.05$

Let $\overline{X}$ and $\overline{Y}$ sample means and ${S_x}^2, {S_y}^2$ unbiased estimators for the variance of 2 independent random samples of size 7 with normal distribution with mean unknown and ...
0
votes
0answers
17 views

unbiased estimator using Walsh Averages?

In class, the professor said the median of the Walsh averages, $\mu _w$, can be used as an estimator. Further $$\dfrac{\textrm{variance of}\;\mu _w}{\textrm{variance of} \;\bar {Y}} =\dfrac{1}{ARE} ...
0
votes
1answer
27 views

If sample standard deviation is biased, why do we use it in typical mean tests? [duplicate]

If sample standard deviation is biased, why do we use it in typical mean tests? Why do we not use an unbiased estimator by dividing the sample standard deviation by C4?
0
votes
0answers
10 views

Expectation of a reciprocal of unbiased estimation [duplicate]

Consider the two "true" value of directions $\bf{n}$, $\bf{s}$ and their "unbiased" estimate ${\bf{n}}_0$ and ${\bf{s}}_0$. Let ${\bf{w}}={\bf{n}}^T{\bf{s}}/{\bf{n}}_0^T{\bf{s}}_0$, then the ...
3
votes
1answer
35 views

Degrees of Freedom In Sample Variance

Recall the formula for sample variance $$s_{n - 1}^2 = \dfrac{1}{n -1} \sum_{i = 1}^n (\bar{x} - x_i)^2,$$ where $\bar{x}$ is the sample mean. There are many proofs for why $s_{n - 1}^2$ is an ...
0
votes
0answers
18 views

Unbiased estimator ( Integral Issue)

$X_1, ... , X_n$ are iid with pdf $f(x|\theta) = e^{-(x-\theta))}I_{(\theta, \infty)}(x)$ it is easy to find the sufficient statistic which is $X_{(1)}$ $E_{\theta}[g(X_{(1)})]=$ $\theta$ (...
1
vote
1answer
44 views

Unbiased Estimator based on Sufficient Statistic

suppose $X_1, ... , X_n$ are iid with pdf $f(x|\beta) = e^{-(x-\beta))}I_{(\beta, \infty)}(x)$ and the pdf of ( the smallest order statistic) $X_{(1)}$ is given by $f_{X_1}(x)$ = n $ *$ $e^{n(\...
0
votes
0answers
15 views

Sampling distribution of the sample standard deviation

I'm trying to recreate the graph from here using equations (2) and (3), but I've yet to do so successfully. I'm not sure that equation (2) is correct. I traced it back to the original source here (...
2
votes
0answers
18 views

Unbiased Estimator of the Standard Deviation of the Sample Standard Deviation

I'm looking for an unbiased estimator of the standard deviation $\text{SD}(s)$ of the sample standard deviation $s = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (x_i - \overline{x})^2}$. I have found this ...
0
votes
0answers
20 views

Unbiased Estimator of the Error Third and Fourth Central Moment in Regression Analysis

In linear regression an unbiased estimator of the error ($\epsilon$) variance is given by: $$MS = \dfrac{1}{n-p-1}\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}$$ which is the sum of squares of ...
3
votes
3answers
114 views

Prove that $\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2}$ is an unbiased estimate of $\text{Var}(\overline{X})$

If $X_{1},X_{2},\ldots,X_{n}$ are independent random variables with common mean $\mu$ and variances $\sigma^{2}_{1},\sigma^{2}_{2},\ldots,\sigma^{2}_{n}$. Prove that \begin{align*} \frac{1}{n(n-1)}\...
1
vote
1answer
46 views

Unbiased estimator for $e^\lambda$ in Poisson distribution

How to generate an unbiased estimator for $e^{-\lambda}$ in Poisson distribution: $\frac{\lambda^k}{k!}{e^{-\lambda}}$ I tried: $$E[a^x]=\sum_{x=0}^\infty a^x\frac{1}{e^{\lambda}}\frac{\lambda^x}{x!}=...
0
votes
1answer
24 views

Good Estimates of the Square of Bernoulli Probability of Success?

I am trying to understand the metrics of a good estimator. For example, the Bernoulli probability of success takes the parameter p. But for X1...Xn iid Ber(p^2) how would you estimate the p^2. How ...
2
votes
1answer
30 views

Standard error of sample variance

We know that an unbiased estimator of the variance is: $$ \hat{\sigma}^2_{unbiased} = \frac{1}{n-1}\sum_{i=1}^n (x_i - \bar{x})^2$$ I was wondering, does it have the smallest possible standard error? ...
1
vote
1answer
51 views

Normal distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample from $X\sim\mathcal{N}(0,\sigma^{2})$. (a) Find the least variance from the set of all unbiased estimators of $\sigma^{2}$. (b) Find a sufficient ...
0
votes
0answers
25 views

Bias/variance of IV estimation

I'm studying IV estimation by myself and have some confusion about the basics. Let $y=X\beta_0 + u$ be a linear model with endogenous variable $X$, and $Z$ be an instrument, meaning that $Z$ and $u$ ...
3
votes
2answers
138 views

95% Confidence interval of $\lambda$ for $X_1,…,X_n$ IID exponential with rate $\lambda$

I know how how to find the estimation of $\hat{\lambda}$ using the method of moments. I can take the first moment and equate it to the empirical to get, $E(X) = \frac{1}{\lambda} = \frac{\sum_{i=1}^{...
4
votes
0answers
33 views

Can we estimate the mean of an asymmetric distribution in an unbiased and robust manner?

Suppose I have i.i.d. samples $X_1, \cdots, X_n$ from some unknown distribution $F$ and I wish to estimate the mean $\mu=\mu(F)$ of that distribution and I insist that the estimator be unbiased - i.e.,...
3
votes
0answers
102 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
4
votes
0answers
76 views

Minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed

What is the minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed? median When we wish to estimate the median, $\mu$, of a normal distributed variable then ...
1
vote
0answers
21 views

Heteroskedasticity-robust White estimator

The heteroskedasticity-robust White estimator is defined as: \begin{align} V_{\hat{\beta}} = (X'X)^{-1}\left(\sum_{i=1}^n x_i x_i' \hat{e}_i^2 \right)(X'X)^{-1} \end{align} with X the matrix of ...
3
votes
2answers
54 views

Where does linear regression fit into the bias-variance tradeoff?

In ISL, the concept of the bias-variance tradeoff is presented with the rule of thumb that simple models will have high bias and that complex models will have high variance. Given this idea, I would ...
0
votes
1answer
44 views

Does minimizing expected squared loss (MSE) result in an unbiased estimator?

I have heard that the estimator with the lowest expected squared loss (mean squared error) is not always unbiased, but I have also heard that the constant that minimizes the expected squared loss vs. ...
3
votes
1answer
164 views

Unbiased estimator of $\lambda(1 - e^\lambda)$ when $x_1,\ldots,x_n$ are i.i.d Poisson($\lambda$)

Suppose $x_1, x_2, x_3,\ldots, x_n$ are i.i.d. random variables with a common Poisson$(\lambda)$ distribution. I was trying to find an unbiased estimator for $\lambda(1 - e^\lambda)$, but I could not ...
0
votes
2answers
36 views

Is there a mistake in the expression of this variance?

I'm busy reading through an econometrics textbook (page 147), and I don't understand the step $$\mathrm {Var}\left(n^{\frac 12}\left(\hat\beta - \beta\right)\right) = \boldsymbol{A^{-1}}\sigma^2\...
1
vote
2answers
64 views

Estimator for $\frac{1}{\lambda}$ using $\min_i X_i$ when $X_i$ are i.i.d $\mathsf{Exp}(\lambda)$

Let $X_1,\ldots,X_n$ be i.i.d. $\mathsf{Exp}(\lambda)$ random variables, where $\lambda$ is unknown. Consider $f_{\min}(x) = \min_{i}(X_i)=$ $ n \lambda $ Exp$(n\lambda x)$. I am told that $\hat \...
2
votes
0answers
12 views

Visualization of unbiasedness of high dimensional paramter estimates

Assume a statistical model $f_{\theta}(X)$ that allows to estimate a parameter vector $\hat{\theta}\in \mathbb{R}^p$ from data $X$ and assume that $p$ is high dimensional (you may assume something ...
1
vote
1answer
89 views

Bootstrap based bias correction

Assume we have a probablistic model $f_{\theta}(x)$ and try to estimate the parameter $\theta$ based on data $x$ with some procedure that yields a biased estimator $$E[\hat{\theta}]=\theta + \eta,$$ ...
2
votes
1answer
42 views

Exogeneity Assumption within or across observations

Suppose we have a linear regression model: $$ y_{i}=x_{i}\beta+\epsilon_{i} $$ Where $i$ is an index for individuals $i=1...N.$ Now, the requirement for unbiased estimation of $\beta$ via OLS ...
0
votes
0answers
69 views

What does it mean that the variance is equal to covariance?

I am reading a paper that says that for forecast unbiasdness it is necessary to asusume that Cov(x,x̂)=Var(x̂), where x̂ is the estimate and x is the true value. How should I interpret this assumption,...
5
votes
1answer
83 views

Variance of unbiased estimator for the shape parameter of Pareto distribution

I'm interested in getting the error bounds of the unbiased estimator of the shape parameter ($\alpha$) using maximum likelihood method of Pareto distribution. The unbiased estimator is known to be ...
1
vote
0answers
12 views

Sample selection in difference-in-differences

I have a dependent variables with a significant amount of zeros (and the share of zeros is different between the control and treatment groups, and changes between the pre- and post-treatment periods). ...
2
votes
0answers
36 views

Why is unbiased estimation from a sample only possible for certain properties?

I was thinking about finding some monotonic measure of entropy of a sample from a continuous distribution (2D, btw), but couldn't think of any such without making assumptions. Why is it that one can ...
3
votes
1answer
92 views

Variance of an Unbiased Estimator for $\sigma^2$

Let $X_1, X_2,...X_n\sim N(0,\sigma^2)$ independently. Define $$Q=\frac{1}{2(n-1)}\sum_{i=1}^{n-1}(X_{i+1}-X_i)^2$$ I already proved that this Q is an unbiased estimator of $\sigma^2$. Now I'm stuck ...
1
vote
1answer
44 views

Distribution function of a biased estimator

$f(y) = ay^{a-1}/θ^a, 0<y<θ$ $ \hat{\Theta} = max(Y_1, Y_2, . . . , Y_n).$ How do I find the $E[\hat{\Theta}]$ ? I'm trying to show that it's a biased estimator, then I'm going to find ...
5
votes
1answer
39 views

Ratio of Unbiased Estimators

If there is a linear regression model as follows: $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + u$$ and we want to estimate the ratio of the slope coefficients: $$\theta = \frac{\beta_1}{\...
2
votes
1answer
55 views

Error of Bias-Corrected Kurtosis Estimators

Background I've found two different bias-corrected estimators for the kurtosis. The first one is used in various software packages, such as MATLAB, and is called bias-corrected in the respective ...
1
vote
0answers
82 views

Consistency vs. Asymptotic Efficiency of estimator

I'm thinking about the relationship between an asymptotically efficient estimator and a consistent estimator, and I'd like to make sure that my thinking is correct. An estimator is asymptotically ...
1
vote
0answers
27 views

Asymptotically unbiased estimator vs consistent estimator [duplicate]

I'm wondering if there is a difference between an asymptotically unbiased estimator and a consistent estimator. For asymptotically unbiased estimators, the expected value of the estimator converges ...
1
vote
0answers
54 views

Determining variance of UMVUE

Let $X_1,...,X_n$ be iid with pdf given by $f(x;\theta)=\frac{log\theta}{\theta^{x-1}}I(x>1)$. My task is to determine if the $\mu=E[X]=1+\frac{1}{log\theta}$ can be estimated efficiently, i.e. if ...
7
votes
1answer
238 views

Are unbiased efficient estimators stochastically dominant over other (median) unbiased estimators?

General description Does an efficient estimator (which has sample variance equal to the Cramér–Rao bound) maximize the probability for being close to the true parameter $\theta$? Say we compare the ...
2
votes
0answers
43 views

Is there any case in which unbiased but larger MSE estimator preferred to biased and smaller MSE one?

Let saying we are interested in a population mean $\mu$ and we have two estimators $\hat{\mu}_{n}^b$ and $\hat{\mu}_{n}^{u}$ defined on $n$ samples such that $\hat{\mu}_{n}^b$ : biased (i.e, $\mathbb{...
6
votes
1answer
211 views

Unbiased estimator of binomial PMF

Is there an unbiased estimator of PMF of a random variable $Y=\sum_{i=1}^{n} X_n $ where $X_i$ are independent Bernoulli trials with probability $p$, that is, the estimator of: \begin{equation}\tag{1} ...
3
votes
2answers
89 views

Unbiased Estimator for $\log\left[\int p(x\mid z)p(z) \, dz\right]$

The naive Monte Carlo estimator is an unbiased estimator for $\int p(x\mid z)p(z) \, dz$, is there a convenient unbiased estimator for $\log \left[\int p(x\mid z)p(z)\,dz \right]$
0
votes
0answers
40 views

Unbiased Estimate of a squared difference between sample of random matrices

In the description below, IER stands for "Inhomogenous Erdos Renyi" random matrix, which is basically saying that the entries in the matrix are Bernoulli distributed and iid. I didn't get the part ...
-1
votes
1answer
121 views

The UMVUE of ratio of parameters for two uniform distributions,

Let $X_1,\ldots,X_m$ be i.i.d. having the uniform distribution $U(0, \theta_x)$ and $Y_1,\ldots, Y_n$ be i.i.d. having the uniform distribution $U(0, \theta_y)$. Suppose that $X_i$’s and $Y_j$’s are ...
0
votes
0answers
25 views

Bias-corrected Property for Jackknife's Pseudo Values

I come across the following formula from a note, saying that we could think of jackknife as a bunch of independent pseudo values with the following form: The notes further comment that the sample ...
1
vote
0answers
26 views

Understanding Rao-Blackwell [duplicate]

From Casella and Berger: Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ ...