Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
2 votes
0 answers
36 views

What are some well-known unbiased estimator of regression coefficient besides OLS estimator?

Is there any other unbiased estimator of regression coefficient than OLS? For instance, one might consider using unbiased estimator with less computational cost (since OLS involves matrix inversion)?
user avatar
0 votes
0 answers
29 views

Unbiased estimator of regression coefficient in high dimension

Is there any unbiased estimator for the regression coefficient $\beta \in \mathbb{R}^p$, p >> 1, where $$ y_k = x_k^T\beta + \epsilon \in \mathbb{R}? $$ Note that $x_k \in \mathbb{R}^p$ and $\...
user avatar
1 vote
1 answer
27 views

Does an endogenous variable bias the coefficient of the exogenous one?

We have the following model: $$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon. $$ We know that: \begin{align*} \operatorname{Cov}(x_1, \epsilon) &\neq 0 \\ \operatorname{Cov}(x_2, \epsilon) &...
user avatar
0 votes
0 answers
19 views

Weighted average estimator - unbiased and consistent

Take an estimator that produces a weighted average of all n observations in an i.i.d sample from a population with mean $\mu$ and variance $\sigma^2$. I.e.: $$ \bar{x}_w = \sum_{i=1}^{n} w_ix_i$$ ...
user avatar
  • 13
0 votes
0 answers
15 views

Is the least mean square estimator for jointly gaussian variables necessarily affine?

In his book on adaptive filtering, Sayed mentions a subclass of affine estimators in which not only the predictions y are linearly dependent on the observations x, but x and y are jointly Gaussian. ...
user avatar
1 vote
0 answers
51 views

Least square estimate expected value and variance of linear model

I am practice some exercises. Here it goes. "Assume we fit the simple model \begin{equation} \hskip 5cmy=X_1\beta_1+\epsilon \hskip 5cm (1) \end{equation} however the true model is \begin{...
user avatar
3 votes
1 answer
36 views

Proof that $g(p)$ unbiasedly estimable only if it is a polynomial (Binomial Distribution)

In Lehmann-Casella (Theory of Point Estimation) they state without proof that if $T \sim Bin(n,p)$, then $g(p)$ is estimable only if it is a polynomial in $p$ of degree $\leq n$. How does one go about ...
user avatar
0 votes
0 answers
11 views

Is the empirical distribution the only unbiased distribution estimator?

Given $n$ samples, if $\hat{p}$ is the empirical distribution of $p \in \Delta_{\mathcal{X}}$ where $\mathcal{X}$ is a finite domain, we know that $\mathbb{E} \hat{p} - p = 0$. Is the empirical ...
user avatar
0 votes
0 answers
42 views

Proportion data: Logistic with MLE vs. OLS with logit-transformed response

This is an expansion of @Beethoven_90's comment on this question. Suppose I have proportion data $Y_i$ computed from a binomial; $Y_i = \frac{S_i}{N_i}$ where $S_i \sim Bin(N_i, p_i)$ and $p_i$ is the ...
user avatar
  • 1
2 votes
0 answers
11 views

Calculating confidence interval for binomial distribution [duplicate]

Suppose we have a sample $X_1, X_2, \ldots, X_n \stackrel{\text{iid}}{\sim} Binomial(\theta)$, where $n$ is known to be large. I would like to calculate the 95% confidence interval for $\theta$, and I ...
user avatar
  • 21
2 votes
0 answers
34 views

Expectation of Difference in Means estimator

Given i.i.d. observations $(Y_i, X_i)$ where $Y_i$ is the response and $X_i$ is binary valued, the difference in means estimator is $$ \hat{\theta} = \frac{1}{n_0} \sum_{i=1, X_i=0} Y_i - \frac{1}{n_1}...
user avatar
  • 1,033
7 votes
1 answer
201 views

Find UMVUE of difference of parameters of two exponential distribution random variables

Let $X_{1}, \dots, X_{n}$ be i.i.d. having the exponential distribution $Exp\left(0, \theta_{x}\right)$ with $\theta_{x}>0$, and $Y_{1}, \dots, Y_{n}$ be i.i.d. having the exponential distribution $...
user avatar
  • 95
1 vote
0 answers
42 views

Why can't OLS estimates be used to obtain regression parameters when dealing with high dimensional data?

Suppose I have a data set consisting of $n$ observations: ${\displaystyle \left\{\mathbf {x} _{i},y_{i}\right\}_{i=1}^{n}}$. If I apply linear regression :${\displaystyle \mathbf {y} =\mathrm {X} {\...
user avatar
0 votes
1 answer
74 views

Proof that multicollinearity doesn't produce biased estimators

I'm trying to prove that multicollinearity doesn't introduce bias into a multiple linear regression model, but my proof seems to indicate the opposite. If we represent the model as $$y = \hat \beta_0 +...
user avatar
0 votes
1 answer
34 views

Will removing a regressor from a model reduce the variance of the remaining regressor

Let's say our full model is a mean centered: $$ y= B_0 + B_1(x_1-\bar x_1) + B_2(x_2-\bar x_2) + e$$ I know $B_0$ works out to be equal to $\bar{y}$, and so $SS_{Reg}(B_0) = 0$ My question is if we ...
user avatar
1 vote
0 answers
25 views

Existence of unbiased estimator for any $f(X)$? [closed]

Suppose you are handed a function of a random variable $f(X)$, how would you construct/rule out the existence of an unbiased estimator for it? I've read through Halmos (1946) but the characterization ...
user avatar
0 votes
0 answers
42 views

Quantifying the bias of a quantile estimator based on order statistics, and its relation to asymptotic unbiasedness

From what I understand, the quantile estimator based on order statistics is asymptotically unbiased (and follows a Normal distribution). I have been looking for a quantification of the non-asymptotic ...
user avatar
  • 1
1 vote
0 answers
29 views

How do we select model for causal inference?

I am reading Rubin's Causal Inference Sec 7.5 in context of completely randomized experiment. It says performing linear regression will produce asymptotically unbiased estimate of causal effect, ...
user avatar
  • 839
2 votes
1 answer
45 views

How to show the Hansen-Hurwitz estimator is unbiased?

Consider a population of size $N$ and draw i.i.d. a random sample $S=(i_1,\dots,i_n)$ of $\{1,...,N\}$ with replacement. We define the Hansen-Hurwitz estimator as $$ \hat{\tau}= \frac{1}{n}\sum_{j=1}^...
user avatar
  • 137
0 votes
0 answers
6 views

Why when the number of data increase the consistency can’t guarantee that the bias induced by the estimator diminishes

Consistency ensures that the bias induced by the estimator decreases as the number of data examples increases. However, the converse is not true asymptotically, an unbiased estimator does not imply ...
user avatar
0 votes
0 answers
14 views

Difference in function mean between two groups with noisy group membership

Suppose that I have a distribution $\mathbb P_X$ over a space $\mathcal X$, and each $x\in \mathcal X$ belongs to either group 0 or group 1, according to a (deterministic) membership function $m: \...
user avatar
  • 215
1 vote
1 answer
33 views

Do robust estimators like M-estimator still have higher variance than OLS in presence of non-normal errors and/or outliers?

In my studies I've learned that even with non-normality of the errors, the OLS estimator is still considered BLUE (Best Linear Unbiased Estimator). The texts also suggested using M and L estimators ...
user avatar
  • 101
1 vote
0 answers
30 views

unbiased estimator and efficiency

can someone plese clarify a doubt for me? Let (X1, . . . , Xn) be a random sample of i.i.d. random variables with expected value $µ$ and variance $σ^2$ Consider the following estimator of $µ$: $T_{n}(...
user avatar
  • 11
0 votes
0 answers
37 views

what is the probability of sample variance when true variance and true mean is unknown?

Sample Variance by definition is $s^2 =\frac{1}{n-1} \sum{(x_i-\bar{x})^2}$ When the population distribution is normal and true variance $\sigma^2$ is known, Sample Variance follows the chisq ...
user avatar
  • 123
0 votes
1 answer
30 views

Can you correct "bias" in a regression if you can measure/model it? A journey in missing data and reweighting test scores

Thank you for joining me on this semi-theoretical journey. Here we will discuss how to account for "predictable" bias in your data. Let's say we have a test composed on many subtests. A ...
user avatar
  • 183
0 votes
0 answers
27 views

what is the bias and variance of this LS estimator?

I want to estimate the variables $a$ and $b$ ($\theta = \left[ {\begin{array}{*{20}{c}} a\\ b \end{array}} \right]$) in the nonlinear model: $$y\left( t \right) = au\left( t \right) + b\exp (u(t)) + ...
user avatar
1 vote
0 answers
36 views

Symmetric distribution with defined mean: is the median always unbiased for the mean?

Let $X_1,\cdots,X_n\overset{iid}{\sim} F_X(x)$ be a random sample from a symmetric distribution with a defined mean. If need be, assume that $n$ is odd and that $F_X(x)$ is continuous. Is it always ...
user avatar
  • 30.8k
0 votes
1 answer
39 views

Unbiased least squares estimate for GM Theorem

In order to prove the Gauss-Markov Theorem, we first have to show that the OLS estimate $\hat{\theta}$ is an unbiased estimator. From what Im reading on Internet and some textbooks, these are the main ...
user avatar
  • 113
4 votes
0 answers
156 views

In Ordinary Least Square (OLS) estimation: is the slope actually an "Inverse-variance weighting" estimator?

I am suspecting the answer is yes, but I'd appreciate help in proving it (even though we know that the estimator is BLUE, so it should probably hold). For context: An Inverse-variance weighting is ...
user avatar
1 vote
0 answers
24 views

How to find an unbiased estimator for reciprocal of scale parameter given an iid exponential sample?

For a random sample $X_1, ..., X_n$ from an exponential distribution with scale parameter $\lambda$, the density is given by $f(x) = \frac{1}{\lambda}e^{-\frac{1}{\lambda}x}; \,x \geq 0,\, \lambda >...
user avatar
  • 584
0 votes
1 answer
116 views

Show unbiased OLS estimator and expression for variance of OLS estimator

Consider the usual linear mixed model: $$Y=X \beta+ZB+\epsilon $$ where Y and $\epsilon$ are $n$-dimensional random variables and $B$ is a $q$-dimensional random variable independent of $\epsilon$ so ...
user avatar
  • 253
0 votes
0 answers
19 views

Correct bias with known DAG

I have the following causal graph: $T \to P$ $(T, P) \to S$ So $T$ causes $P$ (partially) and $T$ and $P$ both cause $S.$ If I just regress $S\,\text{~}\,T + P,$ I will get an overestimated effect for ...
user avatar
0 votes
1 answer
44 views

Unbiased Estimator for Mean Response to Treatment

$\newcommand{\eps}{\varepsilon}\newcommand{\szdp}[1]{\!\left(#1\right)}$ Problem Statement: Consider the following model for the responses measured in a randomized block design containing $b$ blocks ...
user avatar
0 votes
0 answers
24 views

If $T$ is a complete sufficient statistic, then $Cov(T, U)=0$ for all unbiased $U$ [duplicate]

I want to prove the following- Show that if $T$ is complete sufficient for $θ$, then $Cov_θ(T, U) = 0$ for all $θ ∈ Θ$ and for all $U$ satisfying $E_θ(U) = 0$ for all $θ ∈ Θ$. I think in essence it ...
user avatar
0 votes
0 answers
29 views

UMVUE of the following parameter

Suppose I have $\{X_i : 1\le i \le m\}$ which are i.i.d random variables having Poisson distribution with parameter $\lambda$ and let $N_i = \min\{k : X_k > p \text{ and } k \ge i\}$ where $p<\...
user avatar
1 vote
0 answers
42 views

Deriving unbiasedness of estimators (involving method of moments idea) of normal distribution with heterogeneous variance

I am currently reading this paper and in pp.127, 128, there are unbiased estimators that I cannot derive its unbiasedness. The setting is simple. Let $$X_i\sim N(\mu,\tau^2+\sigma_i^2),\quad i\in\{1,\...
user avatar
1 vote
0 answers
63 views

Unbiased estimator of $1/(1-a)$

Let $X_1,...,X_n$ be Poisson with parameter $a$. I am looking for a unbiased estimator of $h(a)=\frac{1}{1-a}$ Let $T$ be a statistic and $g(t)$ be it's pmf. Then if we have $E(T)= h(a)$ then $T$ is ...
user avatar
0 votes
1 answer
90 views

What is the difference between selection bias and composition effect?

Groups we seek to compare (e.g., a treatment group and control group) may differ in ways that constrain our ability to do so. Often, potential outcomes may differ systematically across groups, such as ...
user avatar
0 votes
1 answer
53 views

Property of unbiased estimators

If $f(x)$ and $f(y)$ are both unbiased estimators of $\mu$, aka $E[f(x)]$ = $E[f(y)]$ = $\mu$, is it possible that $f((x+y)/2)$ is also an unbiased estimator of $\mu$? We know $f((x+y)/2)$ would be ...
user avatar
  • 1
1 vote
1 answer
135 views

Unbiasedness of Covariance Matrix Estimator in OLS

I want to prove that $V$ is an unbiased estimator of the covariance matrix $$(X'X)^{-1}(X'DX)(X'X)^{-1},$$ where $D=diag(\sigma^2,...,\sigma^2)=E(ee'|X)$ in a linear model. $$V = \frac{n}{n-k}(X'X)^{-...
user avatar
0 votes
0 answers
31 views

Antithetic variate as control variate to find optimal constant [duplicate]

Problem: If $\hat{θ}_1$ and $\hat{θ}_2$ are unbiased estimators of $θ$, and $\hat{θ}_1$ and $\hat{θ}_2$ are antithetic, we derived that $c^∗ = 1/2$ is the optimal constant that minimizes the variance ...
user avatar
0 votes
0 answers
18 views

STD of response change of linear regression

Let's suppose I have a single predictor linear dependence: $y = kx+b + \epsilon$. A linear regression was performed on the available data and we have the estimates of $\epsilon$, $k$ and $\sigma_k$. ...
user avatar
0 votes
0 answers
19 views

The unbiasedness of OLS estimator under random, zero-mean and independent regressors

Consider the following regression model: y=x'β+z'γ+e, where x and z are p×1 and q×1 zero-mean random vectors consist of regressors respectively, e is the error term, x, z and e are independent of each ...
user avatar
0 votes
0 answers
53 views

UMVUE of $\theta = P(X_1 \leq c)$ [duplicate]

I could use some help as im working through a practice/homework problem. Let Let $X_1, X_2, ... X_n \overset{\text{iid}}\sim N(\mu,1)$, Find the UMVUE of $\theta = P(X_1 \leq c) $ where c is a known ...
user avatar
  • 1
1 vote
1 answer
29 views

Conceptual error in unbiased estimates and their programming in R

A few days ago I asked a question which had a programming error, I already corrected it and I bring it to you again. By generating $n=1,000,000$ of random data with normal distribution ($\mu=35, \...
user avatar
0 votes
0 answers
71 views

Unbiased estimator of minimum order statistic

Let $X_1,X_2$ and $X_3$ be a random sample taken from a continuous population with distribution function F. Consider the function $E(X_{1:3})$ , where $X_{1:3}$ is the minimum order statistic. Can $$...
user avatar
12 votes
5 answers
2k views

Does the biased estimator always have less variance than unbiased one?

Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always. ...
user avatar
  • 279
3 votes
1 answer
130 views

What is an "unbiased forecast"?

Assume we estimate a model from the data $(X, Y)$, with some estimator $W(X, Y)$, which is estimating parameters $\theta$ for the model we chose. Then, we would like to perform a forecast for $Y_h$ ...
user avatar
  • 1,016
3 votes
1 answer
110 views

How to explain intuitively to a lay audience that the variance is an unbiased estimator?

I have data for the concentration of several chemicals in the milk of 10000 cows and have to explain to policymakers and the lay public (i.e. people with no or limited knowledge of statistics) that ...
user avatar
  • 31
0 votes
0 answers
106 views

Variance of a linear estimator

There is a theorem (here, Theorem 3.2.) which says: Let $x_i \sim p_i(\mu_i, \sigma_i^2)$ for $1 \leq i \leq n$ be a set of pairwise uncorrelated random variables. Consider the linear estimator $y_{n,...
user avatar
  • 1

1
2 3 4 5
14