Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...

learn more… | top users | synonyms

4
votes
0answers
108 views

Is the standard explanation for Bessel's correction in unbiased sample variance wrong?

The standard intuition for Bessel's correction $n-1$ instead of $n$ given in many statistical textbooks is something along the lines that you have one degree of freedom fewer because of the presence ...
1
vote
1answer
51 views

Don't understand identity in proof for unbiased sample variance

Wikipedia gives the following proof why to use Bessel's correction for the unbiased sample variance: \begin{align} E[\sigma_y^2] & = E\left[ \frac 1n \sum_{i=1}^n \left(y_i - \frac 1n ...
5
votes
1answer
104 views

What will be the estimator for these parameters

Question: $y_0 = z^d$ is computed from the sum of some recordings by a sensor. Let, there be $k$ sensor nodes. This parameter is calculated by each sensor node and then transmitted to the base ...
7
votes
0answers
156 views

Biased estimator for regression achieving better results than unbiased one in Error In Variables Model

I am working on some syntatic data for Error In Variable model for some research. Currently I have a single independent variable, and I am assuming I know the variance for the true value of the ...
3
votes
1answer
43 views

More than one unbiased estimator for a single unknown parameter?

Is it possible to have more than one unbiased estimator for a single unknown parameter?If "Yes" then how and if "No" the why?
0
votes
0answers
35 views

Unbiased estimator based on minimal sufficient statistic has smaller variance than one based on sufficient statistic

Suppose that $T_1$ is sufficient and $T_2$ is minimal sufficient, U is an unbiased estimator of $\theta$, and define $U_1=\mathbb{E}(U|T_1)$ and $U_2=\mathbb{E}(U|T_2)$ a)Show that ...
19
votes
4answers
981 views

Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?

The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated. $$ ...
0
votes
0answers
23 views

Showing a variance estimator is unbiased

I am trying to show that the variance estimator $ \hat{\sigma}^2 = \sum_{i=1}^{N}(X_{i}^{2}+ X_{i}X_{i-1} + X_{i+1}X_{i})$ is unbiased. $E(\hat{\sigma}^2) = \sigma^2$. I know that ...
2
votes
0answers
39 views

MLE estimate of normal distribution

I am quoting this from Greene's econometrics book: The occasional statement that the properties of the MLE are only optimal in large samples is not true, however. It can be shown that when ...
1
vote
1answer
37 views

Unbiased Estimators

So I've been banging my head against the wall trying to figure out where to go with these problems, and I'm looking for a little direction. Suppose that $Y_1, Y_2, Y_3$ is a random sample where the ...
2
votes
0answers
35 views

Unbiased Estimator for Uniform Distribution

$X_1$ , a sample size 1 is drawn from a uniform distribution over $[0,\theta]$. Find an unbiased estimator for the variance of the population. Find a function for $X_1$, $\tau(X_1)$ such that ...
2
votes
0answers
41 views

Finding MLE with ordered statistics?

Let Y1 < Y2 < ... < Yn be the order statistics of a random sample of size n from the uniform distribution of the continuous type over the closed interval: $$[\theta - \rho, \theta + \rho]$$ ...
-2
votes
1answer
39 views

What is the Mean Squared Error for this estimator?

I understand that to find the MSE, i must find the variance and bias and add them together. I've had trouble calculating either of these so a breakdown would be immensely helpful. The estimator is: ...
1
vote
0answers
26 views

Methods of Proving that a UMVUE does not exist?

Are there efficient methods of showing when a UMVUE does not exist? I can think of the trivial case when no unbiased estimators exist at all. But that's not really interesting. I feel like this ...
0
votes
1answer
46 views

bias and sampling

This was an interview question I encountered. can some one answer this When you sample, what bias are you inflicting? How do you control for biases? What are some of the first things that come to ...
2
votes
0answers
67 views

Where is the maximum bias and variance in a histogram as non-parametric density estimator?

I am a little bit confused about bias and variance of non-parametric density estimators and hope you can help me. Assuming a constant bandwidth and sample size, I am wondering at which points of the ...
0
votes
0answers
38 views

Standard Error for Relative Frequency Distribution

I have a weighted distribution with weights $w_i$, such that: $$\sum_i{{w_i}}=1$$ I know that the mean is defined by: $$\sum_i{{w_i}{x_i}}=\mu$$ And that the unbiased variance is defined as: ...
0
votes
0answers
31 views

Is there a way to check completeness of certain sufficient statistics?

In general, given a p.d.f. or a p.m.f., is there a method to check if a certain statistic is complete? For example, consider a population $N(\theta,1)$ where $\theta$ is unknown and the statistic ...
0
votes
0answers
18 views

ELO Rating for players that do not learn

Lately there are some AI competitions online, where the bot has no capability to "learn/adapt" (ironically) over games. Thus, each uploaded version of a bot by a player is static. On the website, ...
2
votes
0answers
29 views

A Proof of Tukey's Inequality

Suppose that $W_1,W_2,...,W_n$ are uncorrelated unbiased estimators of a parameter $\theta$. Consider $W=\sum_{i=1}^na_iW_i$ such that $E(W)=\theta$ and $Var(W_i)=\sigma^2_i$, where the $a_i$'s are ...
3
votes
1answer
99 views

Finding an unbiased estimator with the smallest variance

I will state the question then my methodology. Q: We have 3 random variables, $X1,X2,X3$ that are independent and identically distributed (iid). We would like to estimate $\theta = E[X1]$. Suppose ...
2
votes
1answer
90 views

Best OLS estimators

Hi i am stuck on this one, the question is related to Gauss-Markov theorem: Consider a general alternative to the OLS estimator that is also a linear unbiased estimator, say ${\tilde \beta}$. ...
13
votes
2answers
301 views

For which distributions is there a closed-form unbiased estimator for the standard deviation?

For the normal distribution, there is an unbiased estimator of the standard deviation given by: $$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} ...
1
vote
0answers
23 views

Unbiased Estimator of Product

Suppose there are stationary times series $\{A_i\}_{i=1}^{T},\{B_i\}_{i=1}^{T},\{C_i\}_{i=1}^{T},\{D_i\}_{i=1}^{T}$, which may not necessarily be independent processes. We know that ...
3
votes
1answer
108 views

Unbiased estimator and sufficient statistic from discrete uniform distribution

$z_1,...z_n$ is a sample from a discrete $\{1,...,N\}$ uniform distribution. I have two questions: 1; I want to find an unbiased estimator for N, with the help of $z_1$ 2; I want to find a 1 ...
1
vote
0answers
56 views

Is Mean Squared Error an unbiased estimator to the error variance?

According to the wikipedia page at http://en.wikipedia.org/wiki/Mean_squared_error, it has: Note that, although the MSE is not an unbiased estimator of the error variance, it is consistent, ...
0
votes
0answers
60 views

Best linear unbiased estimator

I have a sample of N stocks. I have the following information: For each stock i, I have an estimate of variance (of returns) $\hat{\sigma}^2_{i}$. I also have a fitted variance, denoted by ...
0
votes
0answers
28 views

How to estimate the bias of a estimator of an unknown length?

Let's say I have a ruler of unknown length L. I then measure n objects with the ruler, obtaining a sample x1,..xn. If an object is longer than the ruler, I'll register xi=L. I want an unbiased ...
0
votes
1answer
45 views

Is an WLS estimator unbiased, when wrong weights are used?

It is clear that the WLS estimators are consistent if the "wrong" weights used aren't correlated with the explanatory variables. However, I don't know whether this also holds true for unbiasedness.
2
votes
1answer
63 views

Conceptual question on estimation : How to calculate the variance of estimation error

EDIT/ UPDATE: I have understood CRLB & why we need it. But my problem is something else. In book ...
3
votes
1answer
52 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
2
votes
0answers
38 views

Unbiased estimator of the variance [duplicate]

I am going through the book The Elements of Statistical Learning and I'm finding it extremely terse. I have a background in probability but not statistics so perhaps that is why. Anyway, in Chapter 3, ...
1
vote
1answer
86 views

Variance of Mean of Samples from Unknown Distribution

I have a bunch of IID samples from a random variable with unknown mean and unknown variance. I now need to know the variance of the average of these samples. I found some references. In decreasing ...
2
votes
0answers
26 views

If we nonlinearly transform the LS estimates, will they still be unbiased estimates of the true value?

So this is an discussion which came up with a friend/colleague who is a physicist postdoc. He has a bunch of data $(x_i,y_i)$ and wants to fit it to the form $y=e^{ax}$. He uses (weighted) nonlinear ...
1
vote
0answers
15 views

Can someone provide an intuitive explanation of how Stein's Unbiased Risk Estimator can be used for signal denosing?

How does SURE just knows what the noise content of a signal is and how can such an estimator eliminate the noise? Note: I'm not very familiar with estimation theory in statistics
2
votes
1answer
53 views

Calculating the expected value and variance of an estimator of a normal quantile

I don't quite understand how to use the estimator function and the variance function and plug in the sample mean. I expected that we would plug in the value $\bar X - 1.645s$ into $E(s)$ and $V(s)$. ...
1
vote
1answer
109 views

Stein unbiased estimate of risk as surrogate for MSE

I am having trouble understanding Stein's unbiased risk estimate (Stein, C. M. (1981). Estimation of the Mean of a Multivariate Normal Distribution. The Annals of Statistics, 9(6), 1135–1151. ...
3
votes
0answers
59 views

Finding $Var(S^2), E(S^4),$ and unbiased estimator for $\sigma^4$ from random, normal samp

Let $X_1,...,X_n$ be a random sample of size $n$ from the normal distribution $N(\mu,\sigma^2)$ and let $S^2$ be the sample variance. (a) Find $V(S^2)$ and derive $E(S^4)$. (b) find an unbiased ...
1
vote
2answers
78 views

$Var(\bar{X})$ for a random sample from Bernoulli Distribution

Let $X_1,...,X_n$ be a random sample of size $n$ from a Bernoulli distribution with parameter $p$ where $0< p< 1$ is unkown. (a) Find $\theta^2=Var(\bar{X}).$ (b) Find the value of $c$ so that ...
1
vote
0answers
38 views

Is this consistent and/or unbiased?

Does OLS produce inconsistent and/or bias estimators? The first example has the beta raised to a power. $$ y = β_0 + (β_1^2 x_1) + (β_2 x_2) + u $$ The second example has (beta + 1)estimator. $$ y ...
1
vote
1answer
57 views

Finding the unbiased variance estimator in high dimensional spaces

The problem comes from linear regression. Assume the regression function is linear, i.e. $$ f(X) = \beta_0+\sum_{j=1}^pX_j\beta_j $$ .Given a set of training data $(x_1, y_1),\ldots,(x_N,y_N)$,we try ...
0
votes
2answers
53 views

efficiency - bias trade-off

Under which conditions would a researcher choose optimally when there is a trade-off between the variance and bias of an estimator? I hope this question is not too broad... Any help would be ...
3
votes
0answers
87 views

Unbiased estimator with minimum variance for $1/\theta$

Let$ X_1, ...,X_n$ be a random sample feom a distribution $Geometric(\theta)$ for $0<\theta<1$. I.e, $$p_{\theta}(x)=\theta(1-\theta)^{x-1} I_{\{1,2,...\}}(x)$$ Find the unbiased estimator ...
2
votes
2answers
91 views

Obtaining an estimator via Rao-Blackwell theorem

Let $X_1,...,X_n$ be iid with pdf $$f(x|\theta) = exp(\theta -x) I(x)_{(\theta, \infty)}$$ It is asked to find an unbiased estimator for$ \theta $ that is function of a sufficient statistical for ...
2
votes
1answer
129 views

Asymptotically unbiased estimator using MLE

I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across. Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; ...
5
votes
2answers
125 views

How would one formally prove that the OOB error in random forest is unbiased?

I have read this statement many times but have never come across a proof. I would like to try to produce one myself but I'm not even sure on what notation to use. Can anyone help me with this?
1
vote
0answers
61 views

Is it necessary to transform an econometric model in order to have only diagonal elements in the error covariance matrix?

Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $$y_{jis} = x_{jis} \beta + \upsilon_{jis}$$ $j$ refers to school, $i$ refers to ...
24
votes
3answers
2k views

How exactly did statisticians agree to using (n-1) as the unbiased estimator for population variance without simulation?

The formula for computing variance has $(n-1)$ in the denominator: $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$ I've always wondered why. However, reading and watching a few good videos about ...
3
votes
0answers
66 views

Combining unbiased estimators with unknown variance

Say we are given a sequence of independently (but not identically) distributed random variables $X_1,...,X_n$ which are known to be bounded, $X_t \in (a,b)$, and to have the same mean, $\mathbb{E}X_t ...
4
votes
1answer
159 views

Should sampling happen with or without replacement

When doing a simple random sample to estimate population mean for some statistic, how do I know whether sampling happens with or without replacement? It feels wrong to use replacement, because 1) my ...