Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...

learn more… | top users | synonyms

1
vote
0answers
14 views

Variance of an unbiased estimator is 0 when the sample size goes to infinity

So I would like a proof for the following but I can't seem to do it myself. I have a random variable $X$ and I draw $n$ samples($\{X_1, \ldots, X_n\}$) from it and I have $$ Z_n = \frac{\sum_{i = ...
0
votes
0answers
25 views

Estimator for E[A]/E[B]^2

If I have 2 random variables $A$ and $B$, with unknown means (denoted $\mu_A$ and $\mu_B$), and I want to estimate $\mu_A/\mu_B^2$ from $n$ samples of each, $\frac{\sum A_i/n}{(\sum B_i/n)^2}$ is ...
1
vote
0answers
38 views

Find Bayesian estimator with random sample from poisson and prior distribution from exponential distribution

Let $X_{1}$ .....$X_n$ be a random sample from Poisson distribution, prior distribution with parameter $\pi(\lambda) = \beta e^{-\beta\lambda}$. Find Bayesian estimator I couldn't take integration ...
1
vote
1answer
60 views

Using MSE instead of log-loss in logistic regression

Suppose we replace the loss function of the logistic regression (which is normally log-likelihood) with the MSE. That is, still have log odds ratio be a linear function of the parameters, but minimize ...
0
votes
0answers
11 views

Estimate parameter of correlated regressors using two-step regression

Consider the following "true" multiple regression model (observation index omitted for simplicity): $$ Y = \alpha_{1} + \alpha_{2}X + \alpha_{3}Z + u $$ where $u$ is white noise, normally ...
3
votes
1answer
41 views

Sampling distribution of unbiased estimator

Is the sampling distribution of an unbiased estimator symmetrically centered around the true value of the parameter? Why? Why not? Intuitively I think the question above is true (since I can use the ...
3
votes
0answers
32 views

Unbiased estimator variance of sample variance

I was reading the section on k-statistics on wolfram alpha. It was known to me that for the sample variance $k_2 = \frac{1}{n-1}\sum_{i=1}^n (x_i - \overline{x})^2$ it holds that its variance ...
4
votes
0answers
79 views

What's the difference between asymptotic unbiasedness and consistency?

Does each imply the other? If not, does one imply the other? Why/why not? This issue came up in response to a comment on an answer I posted here. Although google searching the relevant terms didn't ...
0
votes
0answers
22 views

Unbiased estimate of a geometric random variable

A coin for which the probability of heads showing up on tossing it is $p$ is tossed $15$ times.The first head appeared in the 3rd toss and 6 heads showed up in $15$ tosses,what are the unbiased ...
6
votes
1answer
50 views

Estimating a variable from its cosine corrupted by additive Gaussian noise

I observe $y_i=\cos(\theta)+z_i$, $i=1,\ldots,n$, where each $z_i\sim\mathcal{N}(0,\sigma^2)$ is an i.i.d. zero-mean Gaussian random variable. I am interested in estimating $\theta\in[0,\pi]$ with ...
0
votes
0answers
21 views

Unbiasedness of the OLS estimators

A general definition of an unbiased estimator is the following: an estimator is unbiased when its probability distribution has an expected value equal to the parameter it is supposed to be estimating. ...
4
votes
1answer
36 views

$Y=\epsilon$ in GLM?

In general linear model $$Y=X\beta +\epsilon $$ the LSE for $\beta$ is $$\hat \beta=(X^TX)^{-1}X^TY$$ and so $$\hat Y=X\hat \beta=X(X^TX)^{-1}X^TY=HY$$ where $H=X(X^TX)^{-1}X^T$. Then the ...
4
votes
1answer
35 views

Why is are unbiased statistics used more commonly than statistics with lower MSE?

I understand the difference between consistency and bias; one converges as the sample size increases, and the other converges as the number of estimates increases, respectively. But, I don't ...
0
votes
0answers
14 views

Unbiased proportion estimate for a 2-component Gaussian mixture model?

Suppose I have a 2-component Gaussian mixture model of dimension $p$, where each mixture has the same covariance matrix: $$ \pi_1 \mathcal{N}(\mu_1, \Sigma) + \pi_0 \mathcal{N}(\mu_0, \Sigma) $$ ...
2
votes
1answer
81 views

Bayesian derivation of unbiased maximum likelihood estimator

I was recently reading an old NIPS paper by Bishop and Qazaz where they claim that an unbiased estimator for variance, based on $N$ Gaussian $\rm i.i.d.$ samples with unknown mean and unknown variance,...
4
votes
0answers
63 views

Is unbiased maximum likelihood estimator always the best unbiased estimator?

I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
25
votes
2answers
987 views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
4
votes
1answer
38 views

Are MVUEs and MLEs always functions of a minimal sufficient statistic?

Is it the case that both minimum variance unbiased estimators (MVUEs) and maximum likelihood estimators (MLEs) are always functions of a minimal sufficient statistic? If so, how do we know? If not, ...
0
votes
0answers
42 views

Why is bias “constant” in bias variance tradeoff derivation?

I know there are plenty of questions about the Bias/Variance tradeoff. I've been trying to derive it myself to build some intuition. I looked at the Wikipedia page, and I saw this: Notice where ...
9
votes
1answer
257 views

Understanding bias-variance tradeoff derivation

I am reading the chapter of bias-variance tradeoff of The elements of statistical learning and I have doubt in the formula at page 29. Let the data arise from a model such that $$ Y = f(x)+\epsilon$$ ...
2
votes
1answer
32 views

Unbiased estimators of a transformed Poisson

So the question I have is$:$ Let $X$ follow a Poisson distribution with mean $\mu$, except that $X=0$ cannot be observed; this gives a random variable $Y$ which has a truncated Poisson ...
1
vote
0answers
38 views

Consistency and Unbiasedness Proof Examples [closed]

I was wondering if you have any suggestion on where to find done proofs of consistency and unbiasedness for estimators (apart from sample mean and variance which is trivial). If you have some it ...
1
vote
1answer
25 views

Determine bias of estimator?

If I have a unbiased estimator, I may obtain another estimator through a reparametrisation. Say, if I have $\lambda$ for a sequences of iid exponentials, I have the unbiased estimator $\sum X/n$, and ...
3
votes
1answer
51 views

OLS Regression : Efficiency of the estimator of the variance of the residuals under the assumption of normality

My question is probably already answered somewhere but I did not find it. In the standard linear regression model under the assumption that residuals are normally distributed, we have a result ...
1
vote
0answers
35 views

Bias in the regression coefficients of a generalized linear model under MLE

Question: Are the regression coefficients of a generalized linear model biased when estimated through maximum likelihood? Imagine, we have a generalized linear model where $E[Y] = g^{-1}(\mu)$ for ...
2
votes
0answers
42 views

The normal and bernoulli distributions

I'm working on this problem and am a little stumped. I was wondering if someone could give me a hint? $x_1,...,x_n$ are iid $N(\mu,\sigma^2)$ where $\mu$ is unknown and $\sigma$ is known. $n>3$ ...
1
vote
1answer
35 views

Linear Regression - Conditions for unbiased estimate

When is the linear regression estimate of $\beta_1$ in the model $$ Y= X_1\beta_1 + \delta$$ unbiased, given that the $(x,y)$ pairs are generated with the following model? $$ Y= X_1\beta_1 + X_2\...
1
vote
1answer
27 views

The projection matrix and proof of an unbiased estimator for sigma-squared

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the ...
1
vote
1answer
35 views

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$. I can't figure this out, finding UMVUE always confuses me. Any help is greatly appreciated
9
votes
2answers
179 views

What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
1
vote
1answer
31 views

Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
2
votes
3answers
63 views

How to Prove Unbiased Estimator

I'm unsure of how to convince myself that $$\hat{\beta} = \frac{\sum X_i Y_i}{\sum X_i^2}$$ is an unbiased estimator when the regression model $$Y_i = \beta X_i + \epsilon_i$$ follows basic OLS ...
0
votes
0answers
43 views

Estimating variance or coefficient of variation for population given total and count for four segments of population

My goal is to estimate the coefficient of variation for a population, but the data we have been given is very limited. All we have is the total and count for four segments of the population. The ...
0
votes
1answer
42 views

If given statistic is unbiased estimator?

I am having trouble finding out and verifying estimators properties ( like unbiased, consistency sufficiency, efficiency ) but in this particular problem given below I have to find a constant such ...
1
vote
0answers
26 views

Regression method if dependent variable is the absolute value of a continous variable [duplicate]

Suppose we have a dependent variable $Y$ that has normal distribution with a mean of $0$. If I run a regression model using the absolute value of $Y$, $|Y|$, i.e. $|Y| = b_1 + b_2 X + u$, my dependent ...
0
votes
1answer
95 views

Unbiased Estimators and Heteroskedasticity

Consider a consumption model with bivariate data points $(Y_i,X_i)$, $i=1,...,n$, with $Y_i$ consumption and $X_i$ income. The univariate model is $$Y_i=\beta X_i+u_i,$$ where $E(u_i|X_i)=0, \textrm{...
2
votes
0answers
34 views

Bias of an estimated Gaussian density

I have an iid sample, $X_1,\dots,X_N \in R^d$, from a multivariate normal density with mean $\mu$ and covariance matrix $\Sigma$. I am estimating the density $p(y) = N(y| \mu, \Sigma)$, using $\hat{...
3
votes
1answer
51 views

Law of Iterated Expectations in Practice

I was wondering if the the following is an example of the law of iterated expectations. Say we observe the entire population of the random variable $X$. Call the mean of the population $\mu$ . ...
0
votes
0answers
12 views

Trouble understanding equations with a mix of variances, expected values and means

Let's assume we don't know the real value $\mu$ of the average of a of random variable $x$. We can find an estimator for the variance using the Bessel correction: $$\widehat{V(x)} = \frac{1}{N-1}\...
2
votes
1answer
127 views

Bias of method of moments estimator for Pareto distribution with known scale parameter

Let $x$ be a Pareto distribution with a known scale parameter $m>0$, i.e. $x\sim f(x|a)=\frac{am^a}{x^{a+1}}, x>a, a>0$ $\mathrm{E}\left[X\right]=\frac{am}{a-1}$ Using method of moments ...
1
vote
0answers
15 views

Best estimate of among groups variance with unequal within groups variances

Goal I have about 100,000 sets of groups. For each set, I would like to measure its among groups variance in order to then make comparisons among sets. Description for each set In each set, I have $...
3
votes
1answer
65 views

Closed formula for D4 constant calculation? (Moving range chart constant)

I need to build a Moving Range Shewhart control chart given a series of observations. In short, I have to calculate the central line and the upper and lower limits as follows ...
1
vote
1answer
197 views

Is this an unbiased estimator?

I'm working in the following problem: Let X be a sample of size = 1 from a Poisson distribution with parameter $\lambda$, and let $h(\lambda) = e^{-3\lambda}$. a.) Check if $T = (-2)^X$ is an ...
5
votes
1answer
52 views

why is $1/n \sum_{i} (X_i -10)^2$ unbiased

Let $\{ X_1,X_2,...,X_n \}$ be n observations randomly drawn from normal distribution with mean $10$ and unknown variance. Prove that the estimator $1/n \sum_{i} (X_i -10)^2$ is unbiased. Why is this ...
0
votes
0answers
15 views

how can I check the bias between two groups

I have a presentation at the firm... using the stata. I want to check is there any bias between unweighted mean and weighted mean. and I already calculated the means between them..by common method ...
1
vote
0answers
58 views

Unbiased estimator for AR($p$) model

Consider an AR($p$) model (assuming zero mean for simplicity): $$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$ The OLS estimator (equivalent to the conditional maximum ...
0
votes
0answers
7 views

Local Fisher information with 0 probability

I have a question about the definition of the Fisher information. Let's say I have just two conditional probabilities $P(\mu=\pm|\theta)$ that sum to $1$. I want to calculate the Fisher information ...
-1
votes
2answers
260 views

a question on negative mean square error

for simple random sampling, i have calculated some mean square errors of many types of ratio estimator. Well, i obtained negative mean square error. Is there a mistake? negative MSE is a normal ...
0
votes
0answers
11 views

How does one show that an extension of an algorithm performs as well or better than the original?

If we have an extension of an algorithm (for example, we take OLS and scale the inputs, and solve it as a reformulated OLS problem), how does one show that the extension performs as well or better ...