Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...

learn more… | top users | synonyms

2
votes
0answers
20 views

Is unbiased maximum likelihood estimator always the best unbiased estimator?

I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
24
votes
2answers
887 views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
4
votes
1answer
34 views

Are MVUEs and MLEs always functions of a minimal sufficient statistic?

Is it the case that both minimum variance unbiased estimators (MVUEs) and maximum likelihood estimators (MLEs) are always functions of a minimal sufficient statistic? If so, how do we know? If not, ...
3
votes
0answers
36 views

Unbiased estimator for the variance of the sample variance

The sample variance of an iid sample $(x_1, \dots, x_n)$ is given by the expression $s^2 = \frac{1}{n-1} \sum_{i = 1}^n (x_i - \overline{x})^2,$ with $\overline x = \frac{1}{n} \sum_{i=1}^n x_i$ the ...
0
votes
0answers
33 views

Why is bias “constant” in bias variance tradeoff derivation?

I know there are plenty of questions about the Bias/Variance tradeoff. I've been trying to derive it myself to build some intuition. I looked at the Wikipedia page, and I saw this: Notice where ...
9
votes
1answer
231 views

Understanding bias-variance tradeoff derivation

I am reading the chapter of bias-variance tradeoff of The elements of statistical learning and I have doubt in the formula at page 29. Let the data arise from a model such that $$ Y = f(x)+\epsilon$$ ...
2
votes
1answer
28 views

Unbiased estimators of a transformed Poisson

So the question I have is$:$ Let $X$ follow a Poisson distribution with mean $\mu$, except that $X=0$ cannot be observed; this gives a random variable $Y$ which has a truncated Poisson ...
1
vote
0answers
26 views

Consistency and Unbiasedness Proof Examples [closed]

I was wondering if you have any suggestion on where to find done proofs of consistency and unbiasedness for estimators (apart from sample mean and variance which is trivial). If you have some it ...
1
vote
1answer
23 views

Determine bias of estimator?

If I have a unbiased estimator, I may obtain another estimator through a reparametrisation. Say, if I have $\lambda$ for a sequences of iid exponentials, I have the unbiased estimator $\sum X/n$, and ...
3
votes
1answer
46 views

OLS Regression : Efficiency of the estimator of the variance of the residuals under the assumption of normality

My question is probably already answered somewhere but I did not find it. In the standard linear regression model under the assumption that residuals are normally distributed, we have a result ...
1
vote
0answers
28 views

Bias in the regression coefficients of a generalized linear model under MLE

Question: Are the regression coefficients of a generalized linear model biased when estimated through maximum likelihood? Imagine, we have a generalized linear model where $E[Y] = g^{-1}(\mu)$ for ...
2
votes
0answers
42 views

The normal and bernoulli distributions

I'm working on this problem and am a little stumped. I was wondering if someone could give me a hint? $x_1,...,x_n$ are iid $N(\mu,\sigma^2)$ where $\mu$ is unknown and $\sigma$ is known. $n>3$ ...
1
vote
1answer
33 views

Linear Regression - Conditions for unbiased estimate

When is the linear regression estimate of $\beta_1$ in the model $$ Y= X_1\beta_1 + \delta$$ unbiased, given that the $(x,y)$ pairs are generated with the following model? $$ Y= X_1\beta_1 + ...
1
vote
1answer
26 views

The projection matrix and proof of an unbiased estimator for sigma-squared

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the ...
1
vote
1answer
34 views

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$. I can't figure this out, finding UMVUE always confuses me. Any help is greatly appreciated
9
votes
2answers
138 views

What is the intuition behind defining completeness in a statistic as being impossible to form an unbiased estimator of $0$ from it?

In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased ...
1
vote
1answer
27 views

Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
1
vote
3answers
56 views

How to Prove Unbiased Estimator

I'm unsure of how to convince myself that $$\hat{\beta} = \frac{\sum X_i Y_i}{\sum X_i^2}$$ is an unbiased estimator when the regression model $$Y_i = \beta X_i + \epsilon_i$$ follows basic OLS ...
0
votes
0answers
26 views

Estimating variance or coefficient of variation for population given total and count for four segments of population

My goal is to estimate the coefficient of variation for a population, but the data we have been given is very limited. All we have is the total and count for four segments of the population. The ...
0
votes
1answer
41 views

If given statistic is unbiased estimator?

I am having trouble finding out and verifying estimators properties ( like unbiased, consistency sufficiency, efficiency ) but in this particular problem given below I have to find a constant such ...
1
vote
0answers
26 views

Regression method if dependent variable is the absolute value of a continous variable [duplicate]

Suppose we have a dependent variable $Y$ that has normal distribution with a mean of $0$. If I run a regression model using the absolute value of $Y$, $|Y|$, i.e. $|Y| = b_1 + b_2 X + u$, my dependent ...
0
votes
1answer
88 views

Unbiased Estimators and Heteroskedasticity

Consider a consumption model with bivariate data points $(Y_i,X_i)$, $i=1,...,n$, with $Y_i$ consumption and $X_i$ income. The univariate model is $$Y_i=\beta X_i+u_i,$$ where $E(u_i|X_i)=0, ...
2
votes
0answers
28 views

Bias of an estimated Gaussian density

I have an iid sample, $X_1,\dots,X_N \in R^d$, from a multivariate normal density with mean $\mu$ and covariance matrix $\Sigma$. I am estimating the density $p(y) = N(y| \mu, \Sigma)$, using ...
3
votes
1answer
50 views

Law of Iterated Expectations in Practice

I was wondering if the the following is an example of the law of iterated expectations. Say we observe the entire population of the random variable $X$. Call the mean of the population $\mu$ . ...
0
votes
0answers
12 views

Trouble understanding equations with a mix of variances, expected values and means

Let's assume we don't know the real value $\mu$ of the average of a of random variable $x$. We can find an estimator for the variance using the Bessel correction: $$\widehat{V(x)} = ...
2
votes
1answer
81 views

Bias of method of moments estimator for Pareto distribution with known scale parameter

Let $x$ be a Pareto distribution with a known scale parameter $m>0$, i.e. $x\sim f(x|a)=\frac{am^a}{x^{a+1}}, x>a, a>0$ $\mathrm{E}\left[X\right]=\frac{am}{a-1}$ Using method of moments ...
1
vote
0answers
13 views

Best estimate of among groups variance with unequal within groups variances

Goal I have about 100,000 sets of groups. For each set, I would like to measure its among groups variance in order to then make comparisons among sets. Description for each set In each set, I have ...
3
votes
1answer
51 views

Closed formula for D4 constant calculation? (Moving range chart constant)

I need to build a Moving Range Shewhart control chart given a series of observations. In short, I have to calculate the central line and the upper and lower limits as follows ...
1
vote
1answer
189 views

Is this an unbiased estimator?

I'm working in the following problem: Let X be a sample of size = 1 from a Poisson distribution with parameter $\lambda$, and let $h(\lambda) = e^{-3\lambda}$. a.) Check if $T = (-2)^X$ is an ...
5
votes
1answer
50 views

why is $1/n \sum_{i} (X_i -10)^2$ unbiased

Let $\{ X_1,X_2,...,X_n \}$ be n observations randomly drawn from normal distribution with mean $10$ and unknown variance. Prove that the estimator $1/n \sum_{i} (X_i -10)^2$ is unbiased. Why is this ...
0
votes
0answers
15 views

how can I check the bias between two groups

I have a presentation at the firm... using the stata. I want to check is there any bias between unweighted mean and weighted mean. and I already calculated the means between them..by common method ...
1
vote
0answers
52 views

Unbiased estimator for AR($p$) model

Consider an AR($p$) model (assuming zero mean for simplicity): $$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$ The OLS estimator (equivalent to the conditional maximum ...
0
votes
0answers
7 views

Local Fisher information with 0 probability

I have a question about the definition of the Fisher information. Let's say I have just two conditional probabilities $P(\mu=\pm|\theta)$ that sum to $1$. I want to calculate the Fisher information ...
-1
votes
2answers
181 views

a question on negative mean square error

for simple random sampling, i have calculated some mean square errors of many types of ratio estimator. Well, i obtained negative mean square error. Is there a mistake? negative MSE is a normal ...
0
votes
0answers
11 views

How does one show that an extension of an algorithm performs as well or better than the original?

If we have an extension of an algorithm (for example, we take OLS and scale the inputs, and solve it as a reformulated OLS problem), how does one show that the extension performs as well or better ...
1
vote
2answers
60 views

showing the estimator of $\sigma^2$ [closed]

Let's take a one way ANOVA Model as $Y_{ij}=\mu + \tau_i +\epsilon_{ij}$ $i=1,2,3$ and $j=1,...,n_i$ and $\epsilon_{ij} \sim N(0,\sigma^2), \ \ \ \forall (i,j).$ Sample variance for factor ...
0
votes
0answers
20 views

Pandey and Dubey estimator.

I am studying sampling theory Pandey and Dubey(1988) proposed the following product estimator. $\bar y_{PD} = \bar y \left( \frac{\bar x + C_x}{\bar X +C_x}\right)$ And its Mean square error is ...
0
votes
1answer
23 views

Using the MSE of $c\theta$ to infer $MSE_{c \theta_{hat}}=MSE_{\theta_{hat}}/1+\lambda$

Suppose $\newcommand{\Bias}{\rm Bias}\hat\theta$ is an unbiased estimator of $\theta$ and $\operatorname{Var} \hat\theta =\lambda \theta^2$ for constant $\lambda$. Find the MSE of c$\theta$, where ...
1
vote
1answer
31 views

Estimating $X\sim \mathcal{N}(0,\sigma^2)$ given a bunch of observations with correlated noise

How is the minimum variance unbiased estimator derived for some RV $X\sim \mathcal{N}(0,\sigma^2)$, given $\mathbf{Y}=X\cdot \mathrm{ones}_{N\times 1}+\mathbf{w}$, where $\mathbf{w}$ is ...
1
vote
1answer
44 views

Defintion of the terms “node weight” and “case weight”

In the literature about decision tress and especially the family of tree approaches that avoid selection bias (conditional inference trees e.g. here: ctree: Conditional Inference Trees by Hothorn, ...
0
votes
0answers
15 views

Is OLS unbiased for two subpopulations?

I have a sample of size N. I know that the relationship Y=X*b always holds without error, where Y and X are vectors of size (Nx1). I do not observe X. However, for a fraction theta of the sample, I ...
2
votes
1answer
60 views

For $Y \sim N_n(X\theta,\sigma^2I_n)$ show that an unbiased linear estimator, $b'Y$, of $c'\theta$ is independent of $Y'(I-P_x)Y$

For $Y \sim N_n(X\theta,\sigma^2I_n)$ show that an unbiased linear estimator, $b'Y$, of $c'\theta$ is independent of $Y'(I-P_x)Y$ Ok...first some notation: $'$ means transpose and $P_x$ is the ...
0
votes
0answers
155 views

Unbiased estimator of variance of normal distribution

I am struggling with the following question about unbiased estimators. I don't know if its the wording, but I would appreciate any guidance. Question: Let $X_1,...,X_n$ denote a random sample from a ...
0
votes
0answers
12 views

Using Horvitz-Thompson for estimation from a simple random sample with unknown membership probability

I learnt about the Horvitz-Thompson estimator yesterday, and am trying to apply it to the degenerate case where $p$ is uniform for each, but I seem to have run into a bit of a confusing situation. ...
1
vote
0answers
69 views

Find an unbiased estimator from a random sample from the logistic distribution

So I'm not sure where to start on this one, so any hints will help me because I'm a little lost on finding unbiased estimators. So suppose $X_1, X_2,... X_n$ is a random sample from the logistic ...
0
votes
1answer
65 views

Odds ratio and bias

I have a random sample of size n that are distributed as Bernoulli random variables with parameter $p$. Given $$v=\frac{p}{1-p}$$ we are asked to find the the MLE of $v$, say $\hat{v}$. Further, we ...
1
vote
1answer
29 views

Homoskedasticity and bias in regression

My teacher said "Violations of homoskedasticity assumption does not lead to bias in the estimated coef." Can someone motivate/explain this, perhaps with a formula?
1
vote
0answers
24 views

Asymptotics of the MLE: a different flavor of proof? [Reference request]

I'm currently trying to understand more about the properties of the maximum likelihood estimator. It's known that, in the large data-limit, the MLE becomes an unbiased estimator with almost Gaussian ...
8
votes
1answer
142 views

What is the variance of this estimator

I want to estimate the mean of a function f, i.e. $$E_{X,Y}[f(X,Y)]$$ where $X$ and $Y$ are independent random variables. I have samples of f but not iid: There are iid samples for $Y_1,Y_2,\dots Y_n$ ...