Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...

learn more… | top users | synonyms

5
votes
1answer
36 views

Estimating a variable from its cosine corrupted by additive Gaussian noise

I observe $y_i=\cos(\theta)+z_i$, $i=1,\ldots,n$, where each $z_i\sim\mathcal{N}(0,\sigma^2)$ is an i.i.d. zero-mean Gaussian random variable. I am interested in estimating $\theta\in[0,\pi]$ with ...
0
votes
0answers
20 views

Unbiasedness of the OLS estimators

A general definition of an unbiased estimator is the following: an estimator is unbiased when its probability distribution has an expected value equal to the parameter it is supposed to be estimating. ...
4
votes
1answer
35 views

$Y=\epsilon$ in GLM?

In general linear model $$Y=X\beta +\epsilon $$ the LSE for $\beta$ is $$\hat \beta=(X^TX)^{-1}X^TY$$ and so $$\hat Y=X\hat \beta=X(X^TX)^{-1}X^TY=HY$$ where $H=X(X^TX)^{-1}X^T$. Then the ...
4
votes
1answer
31 views

Why is are unbiased statistics used more commonly than statistics with lower MSE?

I understand the difference between consistency and bias; one converges as the sample size increases, and the other converges as the number of estimates increases, respectively. But, I don't ...
0
votes
0answers
12 views

Unbiased proportion estimate for a 2-component Gaussian mixture model?

Suppose I have a 2-component Gaussian mixture model of dimension $p$, where each mixture has the same covariance matrix: $$ \pi_1 \mathcal{N}(\mu_1, \Sigma) + \pi_0 \mathcal{N}(\mu_0, \Sigma) $$ ...
2
votes
1answer
75 views

Bayesian derivation of unbiased maximum likelihood estimator

I was recently reading an old NIPS paper by Bishop and Qazaz where they claim that an unbiased estimator for variance, based on $N$ Gaussian $\rm i.i.d.$ samples with unknown mean and unknown ...
4
votes
0answers
58 views

Is unbiased maximum likelihood estimator always the best unbiased estimator?

I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
24
votes
2answers
962 views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
4
votes
1answer
36 views

Are MVUEs and MLEs always functions of a minimal sufficient statistic?

Is it the case that both minimum variance unbiased estimators (MVUEs) and maximum likelihood estimators (MLEs) are always functions of a minimal sufficient statistic? If so, how do we know? If not, ...
3
votes
0answers
39 views

Unbiased estimator for the variance of the sample variance

The sample variance of an iid sample $(x_1, \dots, x_n)$ is given by the expression $s^2 = \frac{1}{n-1} \sum_{i = 1}^n (x_i - \overline{x})^2,$ with $\overline x = \frac{1}{n} \sum_{i=1}^n x_i$ the ...
0
votes
0answers
36 views

Why is bias “constant” in bias variance tradeoff derivation?

I know there are plenty of questions about the Bias/Variance tradeoff. I've been trying to derive it myself to build some intuition. I looked at the Wikipedia page, and I saw this: Notice where ...
9
votes
1answer
249 views

Understanding bias-variance tradeoff derivation

I am reading the chapter of bias-variance tradeoff of The elements of statistical learning and I have doubt in the formula at page 29. Let the data arise from a model such that $$ Y = f(x)+\epsilon$$ ...
2
votes
1answer
30 views

Unbiased estimators of a transformed Poisson

So the question I have is$:$ Let $X$ follow a Poisson distribution with mean $\mu$, except that $X=0$ cannot be observed; this gives a random variable $Y$ which has a truncated Poisson ...
1
vote
0answers
34 views

Consistency and Unbiasedness Proof Examples [closed]

I was wondering if you have any suggestion on where to find done proofs of consistency and unbiasedness for estimators (apart from sample mean and variance which is trivial). If you have some it ...
1
vote
1answer
24 views

Determine bias of estimator?

If I have a unbiased estimator, I may obtain another estimator through a reparametrisation. Say, if I have $\lambda$ for a sequences of iid exponentials, I have the unbiased estimator $\sum X/n$, and ...
3
votes
1answer
49 views

OLS Regression : Efficiency of the estimator of the variance of the residuals under the assumption of normality

My question is probably already answered somewhere but I did not find it. In the standard linear regression model under the assumption that residuals are normally distributed, we have a result ...
1
vote
0answers
31 views

Bias in the regression coefficients of a generalized linear model under MLE

Question: Are the regression coefficients of a generalized linear model biased when estimated through maximum likelihood? Imagine, we have a generalized linear model where $E[Y] = g^{-1}(\mu)$ for ...
2
votes
0answers
42 views

The normal and bernoulli distributions

I'm working on this problem and am a little stumped. I was wondering if someone could give me a hint? $x_1,...,x_n$ are iid $N(\mu,\sigma^2)$ where $\mu$ is unknown and $\sigma$ is known. $n>3$ ...
1
vote
1answer
33 views

Linear Regression - Conditions for unbiased estimate

When is the linear regression estimate of $\beta_1$ in the model $$ Y= X_1\beta_1 + \delta$$ unbiased, given that the $(x,y)$ pairs are generated with the following model? $$ Y= X_1\beta_1 + ...
1
vote
1answer
26 views

The projection matrix and proof of an unbiased estimator for sigma-squared

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the ...
1
vote
1answer
35 views

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$. I can't figure this out, finding UMVUE always confuses me. Any help is greatly appreciated
9
votes
2answers
152 views

What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
1
vote
1answer
30 views

Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
2
votes
3answers
62 views

How to Prove Unbiased Estimator

I'm unsure of how to convince myself that $$\hat{\beta} = \frac{\sum X_i Y_i}{\sum X_i^2}$$ is an unbiased estimator when the regression model $$Y_i = \beta X_i + \epsilon_i$$ follows basic OLS ...
0
votes
0answers
33 views

Estimating variance or coefficient of variation for population given total and count for four segments of population

My goal is to estimate the coefficient of variation for a population, but the data we have been given is very limited. All we have is the total and count for four segments of the population. The ...
0
votes
1answer
42 views

If given statistic is unbiased estimator?

I am having trouble finding out and verifying estimators properties ( like unbiased, consistency sufficiency, efficiency ) but in this particular problem given below I have to find a constant such ...
1
vote
0answers
26 views

Regression method if dependent variable is the absolute value of a continous variable [duplicate]

Suppose we have a dependent variable $Y$ that has normal distribution with a mean of $0$. If I run a regression model using the absolute value of $Y$, $|Y|$, i.e. $|Y| = b_1 + b_2 X + u$, my dependent ...
0
votes
1answer
95 views

Unbiased Estimators and Heteroskedasticity

Consider a consumption model with bivariate data points $(Y_i,X_i)$, $i=1,...,n$, with $Y_i$ consumption and $X_i$ income. The univariate model is $$Y_i=\beta X_i+u_i,$$ where $E(u_i|X_i)=0, ...
2
votes
0answers
31 views

Bias of an estimated Gaussian density

I have an iid sample, $X_1,\dots,X_N \in R^d$, from a multivariate normal density with mean $\mu$ and covariance matrix $\Sigma$. I am estimating the density $p(y) = N(y| \mu, \Sigma)$, using ...
3
votes
1answer
51 views

Law of Iterated Expectations in Practice

I was wondering if the the following is an example of the law of iterated expectations. Say we observe the entire population of the random variable $X$. Call the mean of the population $\mu$ . ...
0
votes
0answers
12 views

Trouble understanding equations with a mix of variances, expected values and means

Let's assume we don't know the real value $\mu$ of the average of a of random variable $x$. We can find an estimator for the variance using the Bessel correction: $$\widehat{V(x)} = ...
2
votes
1answer
116 views

Bias of method of moments estimator for Pareto distribution with known scale parameter

Let $x$ be a Pareto distribution with a known scale parameter $m>0$, i.e. $x\sim f(x|a)=\frac{am^a}{x^{a+1}}, x>a, a>0$ $\mathrm{E}\left[X\right]=\frac{am}{a-1}$ Using method of moments ...
1
vote
0answers
15 views

Best estimate of among groups variance with unequal within groups variances

Goal I have about 100,000 sets of groups. For each set, I would like to measure its among groups variance in order to then make comparisons among sets. Description for each set In each set, I have ...
3
votes
1answer
56 views

Closed formula for D4 constant calculation? (Moving range chart constant)

I need to build a Moving Range Shewhart control chart given a series of observations. In short, I have to calculate the central line and the upper and lower limits as follows ...
1
vote
1answer
193 views

Is this an unbiased estimator?

I'm working in the following problem: Let X be a sample of size = 1 from a Poisson distribution with parameter $\lambda$, and let $h(\lambda) = e^{-3\lambda}$. a.) Check if $T = (-2)^X$ is an ...
5
votes
1answer
50 views

why is $1/n \sum_{i} (X_i -10)^2$ unbiased

Let $\{ X_1,X_2,...,X_n \}$ be n observations randomly drawn from normal distribution with mean $10$ and unknown variance. Prove that the estimator $1/n \sum_{i} (X_i -10)^2$ is unbiased. Why is this ...
0
votes
0answers
15 views

how can I check the bias between two groups

I have a presentation at the firm... using the stata. I want to check is there any bias between unweighted mean and weighted mean. and I already calculated the means between them..by common method ...
1
vote
0answers
54 views

Unbiased estimator for AR($p$) model

Consider an AR($p$) model (assuming zero mean for simplicity): $$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$ The OLS estimator (equivalent to the conditional maximum ...
0
votes
0answers
7 views

Local Fisher information with 0 probability

I have a question about the definition of the Fisher information. Let's say I have just two conditional probabilities $P(\mu=\pm|\theta)$ that sum to $1$. I want to calculate the Fisher information ...
-1
votes
2answers
218 views

a question on negative mean square error

for simple random sampling, i have calculated some mean square errors of many types of ratio estimator. Well, i obtained negative mean square error. Is there a mistake? negative MSE is a normal ...
0
votes
0answers
11 views

How does one show that an extension of an algorithm performs as well or better than the original?

If we have an extension of an algorithm (for example, we take OLS and scale the inputs, and solve it as a reformulated OLS problem), how does one show that the extension performs as well or better ...
1
vote
2answers
61 views

showing the estimator of $\sigma^2$ [closed]

Let's take a one way ANOVA Model as $Y_{ij}=\mu + \tau_i +\epsilon_{ij}$ $i=1,2,3$ and $j=1,...,n_i$ and $\epsilon_{ij} \sim N(0,\sigma^2), \ \ \ \forall (i,j).$ Sample variance for factor ...
0
votes
0answers
20 views

Pandey and Dubey estimator.

I am studying sampling theory Pandey and Dubey(1988) proposed the following product estimator. $\bar y_{PD} = \bar y \left( \frac{\bar x + C_x}{\bar X +C_x}\right)$ And its Mean square error is ...
0
votes
1answer
23 views

Using the MSE of $c\theta$ to infer $MSE_{c \theta_{hat}}=MSE_{\theta_{hat}}/1+\lambda$

Suppose $\newcommand{\Bias}{\rm Bias}\hat\theta$ is an unbiased estimator of $\theta$ and $\operatorname{Var} \hat\theta =\lambda \theta^2$ for constant $\lambda$. Find the MSE of c$\theta$, where ...
1
vote
1answer
32 views

Estimating $X\sim \mathcal{N}(0,\sigma^2)$ given a bunch of observations with correlated noise

How is the minimum variance unbiased estimator derived for some RV $X\sim \mathcal{N}(0,\sigma^2)$, given $\mathbf{Y}=X\cdot \mathrm{ones}_{N\times 1}+\mathbf{w}$, where $\mathbf{w}$ is ...
1
vote
1answer
50 views

Defintion of the terms “node weight” and “case weight”

In the literature about decision tress and especially the family of tree approaches that avoid selection bias (conditional inference trees e.g. here: ctree: Conditional Inference Trees by Hothorn, ...
0
votes
0answers
15 views

Is OLS unbiased for two subpopulations?

I have a sample of size N. I know that the relationship Y=X*b always holds without error, where Y and X are vectors of size (Nx1). I do not observe X. However, for a fraction theta of the sample, I ...
2
votes
1answer
62 views

For $Y \sim N_n(X\theta,\sigma^2I_n)$ show that an unbiased linear estimator, $b'Y$, of $c'\theta$ is independent of $Y'(I-P_x)Y$

For $Y \sim N_n(X\theta,\sigma^2I_n)$ show that an unbiased linear estimator, $b'Y$, of $c'\theta$ is independent of $Y'(I-P_x)Y$ Ok...first some notation: $'$ means transpose and $P_x$ is the ...
0
votes
0answers
176 views

Unbiased estimator of variance of normal distribution

I am struggling with the following question about unbiased estimators. I don't know if its the wording, but I would appreciate any guidance. Question: Let $X_1,...,X_n$ denote a random sample from a ...