Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
738
questions
5
votes
0
answers
42
views
No unbiased estimator of $\min\{\mu_1,\mu_2\}$
The problem is stated as:
Suppose $X, Y$ are independent and $X \sim \mathcal{N}(\mu_1, 1), Y \sim \mathcal{N}(\mu_2, 1)$ with unknown parameters $\mu_1, \mu_2$. Prove that unbiased estimation of $\...
0
votes
1
answer
33
views
Example when globally unbiased estimator does not exist while locally unbiased estimator exists?
The locally unbiased(l.u.) estimator $\hat{\theta}\left( x \right)$, with $x$ stands for the experiment result, refers to the estimator that satisfies(see Eq(5) of this paper for multiparameter case) $...
- 85
0
votes
0
answers
49
views
How can I compute the expected value and variance of the 4th power of the sample median?
Given the following parameter estimate, how do I find $E[\hat{a}_{MED}]$ and $Var[\hat{a}_{MED}]$?
\begin{equation}
\label{eq:Estimator_a_Med}
\hat{a}_{MED} = - \left( n_0 \right)^4 \cdot \log(0.5)...
0
votes
0
answers
24
views
Deriving Properties of Estimators (Bias and Variance)
I have the following probability distribution function given by:
\begin{equation}
\label{eq:function}
f(x) = \frac{4a}{x^5} \exp \left[ {- \frac{a}{x^4}} \right] \quad \quad 0 \leq x \leq \infty \...
1
vote
1
answer
54
views
Find a function of $\theta$ so that there exists an unbiased estimator and the variance coincides with Cramér-Rao lower bound
Let $X_1,\dots, X_n$ be a random sample from the geometric distribution $P(X=x)=\theta(1-\theta)^x$ for $x=0,1,2,\dots$ where $0<\theta<1$.
Find a function of $\theta$, say $\tau=h(\theta)$ so ...
- 611
2
votes
0
answers
85
views
Why is i.i.d. an OLS assumption?
Assume the following linear relationship:
$Y_i = \beta_0 + \beta_1 X_i + u_i$, where $Y_i$ is the dependent variable, $X_i$ a single independent variable and $u_i$ the error term.
According to Stock &...
- 345
1
vote
0
answers
46
views
Why isn't an unbiased Standard Error of the Mean calculated with $n-1$? [duplicate]
The unbaised variance of a population from a single sample ($n\ll N$), $s^2=\sum_i(x_i-\bar{x})/(n_x-1)$. ($n_x$ being the sample size.) However, the standard error of the mean: $SE=s/\sqrt{n_m}$, not ...
- 227
0
votes
0
answers
14
views
Fitting the conditional expectation?
Say we want to fit some model to predict $\mathbf{E}(A | B)$, which is the expected value for some distribution (ex. Poisson).
What would be the benefit/loss of fitting this vs. computing $\mathbf{E}(...
- 249
0
votes
1
answer
54
views
Consistent or inconsistent estimator
If $\hat{\theta}_n$ is an estimator for the parameter $\theta$, then the two sufficient conditions to ensure consistency of $\hat{\theta}_n$ are:
Bias($\hat{\theta}_n)\to 0$ and Var$(\hat{\theta}_n)\...
1
vote
0
answers
14
views
Do I need statistical test with nested-cv?
I'm working with a small data set and with 4 algorithms. The optimization process showed to be Very important as long as it improves a lot their performances. So, I'm using 10x5 nested-cv to estimate ...
0
votes
0
answers
46
views
Trouble understanding expected value (does it assume infinite sample size?) and bias vs consistent
This might be a dumb question.. but I was wondering if someone can help me out with the concept expectation. This question started from trying to understand bias vs consistent. So when we roll a dice, ...
- 55
0
votes
0
answers
46
views
Can variance be calculated with a sample size of n=1?
I am currently analysing inter-rater reliability/agreement data for a single case with multiple raters. For that I am using Gwet's $AC_2$ (as described here) using the irrCAC package in ...
- 85
3
votes
0
answers
59
views
Estimating the Absolute Difference in Mean Between Two Normal RVs
I've seen more general questions of this nature, but none that discuss specifically the problem setup that I'm interested in, which I believe is substantially simpler.
Suppose that we observe two ...
- 41
0
votes
0
answers
64
views
Distribution of $S = \min\{ X_1, ..., X_n\}$ and $c$ s.t. $E(c\hat{\theta}) = \theta$
Warning
The question is the third and last part of this question.
Exercise
Let $X \thicksim Pa(\lambda,\theta)$ with density function:
$ f(x; \theta, \lambda) = \frac{\lambda \theta^{\lambda}}{x^{\...
- 73
2
votes
1
answer
63
views
Is AUROC sample size-unbiased
Some metrics are sample size-biased, i.e., they will give biased estimates in small sample sizes. Is there a formal proof that AUROC is either sample size-biased or unbiased? I.e., will the ...
- 1,866
1
vote
1
answer
60
views
Can a “reverse unbiased” estimator be created?
Suppose we have a parameter $\theta$ that we want to estimate. We sample an observation (random variable) $X$ from a known distribution $D_{X|\theta}$. Then, we can compute an estimator $\hat\theta(X)$...
0
votes
1
answer
33
views
Why isn't the pseudo-inverse the best choice in my linear estimation problem?
Context:
I have a problem of the following form. Let $\boldsymbol{\theta}\in\mathbb{R}^n$ be a fixed vector I want to estimate. Let $\mathbf{M}\in\mathbb{R}^{m\times n}$ be a matrix with $m>n$ and $...
- 133
0
votes
0
answers
11
views
Unbiased estimator from a random sample (Poisson) [duplicate]
"Suppose that $X_1,X_2, \cdots, X_n$ is a random sample from a Poisson distribution with parameter $\lambda$; propose an unbiased estimator for $\theta = exp(-\lambda)$. "
Hello everyone, I'...
0
votes
0
answers
32
views
Will taking mean of sample quantiles be an unbiased estimate the population quantile?
Say I have $N$ samples of 100 numbers all drawn IID from the same distribution $\mathcal{D}$. For each sample, I take the 95th quantile to get $N$ sample quantiles $\hat{q}_n$. Will taking the average ...
- 231
0
votes
0
answers
34
views
Confused about UMVE and Linear Combination of Two Unbiased Estimators, Taking Unif (0, $\theta$) as an Example
A dumb question here ...
I am confused about UMVE and Linear Combination of Two Unbiased Estimator (that can potentially create a more efficient/lower-variance estimator).
Taking $\mathcal{Unif}(0, \...
DumbMLE
2
votes
1
answer
71
views
Multivariate distribution of linear regression coefficients and unbiased variance estimator
Excerpt from "Elements of Statistical Learning", p.47
Assume that the conditional expectation of $Y$ is linear in $X_1, \ldots, X_p$. Also assume that the deviations of $Y$ around its ...
- 165
0
votes
0
answers
33
views
Stratified sampling where observed values vary with strata inclusion
Problem set-up:
Let's say I make $n$ observations of a numerical-scale variable, $x$, where each observation can be cross-classified into $L$ strata. Each observation may belong to anywhere from 1 to $...
- 51
1
vote
1
answer
59
views
Is $\sum(x^2 - \bar{x}^2)/(n-1)$ an unbiased estimator of variance?
We know that sample variance $\sum(x- \bar{x})^2/(n-1) = \sum(x^2- 2x\bar{x}+\bar{x}^2)/(n-1)$ is an unbiasedd estimator of variance
Is $\sum(x^2 - \bar{x}^2)/(n-1)$ also an unbiased estimator of ...
9
votes
5
answers
802
views
How do we know the true value of a parameter, in order to check estimator properties?
For example, we say that an estimator is unbiased if the expected value of the estimator is the true value of the parameter we're trying to estimate. However, if we already know the true value of the ...
1
vote
0
answers
17
views
Offset variable in a Poisson regression measured with error
It is known that measurement error in predictors lead to a bias toward zero, and also bias other predictors’ estimates:
The effect of covariate measurement error on coefficients in regression with ...
- 1,072
4
votes
2
answers
66
views
Are these moments estimators asymptotic unbiased?
In this paper, authors consider method of moments of fitting Gumbel distribution:
We know that maximal likelihood estimators are asymptotic unbiased. But are these moments estimators asymptotic ...
- 611
0
votes
0
answers
35
views
Unbiased estimators for uniform distribution
I have a given question that I just cannot figure out. From what I can understand, both estimators would be unbiased (since e_1 is the sample mean and e_2 is an unbiased estimator for uniform ...
6
votes
3
answers
507
views
Intuition behind a 0% central/equal-tailed confidence interval
At a vague intuitional level, I feel like, if we generated a 0%, two-tailed, truly central (or "equal-tailed") confidence interval (CI), the number it would wrap around would be a median-...
- 775
9
votes
1
answer
168
views
"Unbiased" (at least ballpark) Estimate of Condition Number of True Covariance Matrix being Estimated & other Symmetric Matrices (e.g.,Hessian)
Are there any known ways of getting an unbiased estimate of the condition number of the true covariance matrix being estimated, or at least correct within a small number of orders of magnitude? For ...
- 12.7k
1
vote
0
answers
16
views
Estimating the ratio between the square of the sample mean and the second moment of the sample mean
Suppose $y_1,\dots,y_n$ are i.i.d. and have the $N(\mu, \sigma^2)$ distribution. I'm interested in finding an unbiased estimator for
$$
\rho := \frac{\mu^2}{\mu^2 + \sigma^2/n}.
$$
A naive approach
...
- 11
3
votes
0
answers
12
views
If the bias of an estimator is expressed as a difference, what do you call the ratio of the estimator and true value?
If $Bias(\hat{\beta}) = (\beta - \mathbb{E}[\hat{\beta}])$, is there a term to describe the quantity $\frac{\mathbb{E}[\hat{\beta}]}{\beta}$?
- 568
3
votes
0
answers
31
views
When is it better to have an unbiased estimator instead of one that has a smaller risk?
I just learned that for $X_1, \ldots X_n \sim N(\mu, \sigma^2)$ i.i.d, the sample variance $\frac{1}{n-1} \sum_{i=1}^n (X_i - \bar X)^2$ is unbiased, and it is in fact UMVUE.
However, it is not ...
- 514
1
vote
1
answer
50
views
Estimator of the variance of a population from different groups
Suppose a population can be divided into 4 different groups (A, B, C, D) and you take the sample variance for a parameter of each group.
A proposed estimator for the population variance is
$$s^2_A/n_A ...
0
votes
0
answers
130
views
How to prove the MLE for Weibull distribution is biased or unbiased?
My attempt is like this:
Let $X \sim \text{Weibull}(\alpha, \lambda)$ be a random variable following a Weibull distribution with pdf
$fx(x; \lambda) = \begin{cases}
{\alpha}{\lambda^{-\alpha}} x^{\...
- 1
1
vote
2
answers
46
views
Where do I make a mistake in the proof of the sample variance for the unbiased estimator?
I am trying to prove that the sample variance formula is unbiased.
Firts, let $\mathbb{E}(X_i) = \mu, \text{Var}(X_i)=\sigma^2=\mathbb{E}(X_i^2)-\mathbb{E}(X_i)^2, \bar{X}=\frac{\sum X_i}{n}$.
$\...
- 183
2
votes
1
answer
94
views
Attainablility of Cramer Rao Bound with function of multi-parameters?
Suppose we have multivariables ${\boldsymbol {\theta }}=\left[\theta _{1},\theta _{2},\dots ,\theta _{d}\right]^{T}\in {\mathbb {R}}^{d}$, and we want to estimate the function of parameters $\...
- 85
0
votes
1
answer
41
views
Derivation of the formula for the asymptotic relative efficiency of two estimators with different estimands
Background
In their book, Huber & Ronchetti (pp. 2-3) compare the efficiency of the mean absolute deviation $d_n$ with the standard deviation $s_n$ with the following formula:
$$
\operatorname{ARE}...
- 27.5k
0
votes
0
answers
125
views
Showing the unbiased estimator of variance for GLS estimator
I have the following regression
$$y = X\beta +u$$
where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
- 890
0
votes
0
answers
28
views
Find UMVUE of Ratio of two parametric functions
Let T be UMVUE of $g(\theta)$ and S be UMVUE of $h(\theta)$.
Is there any way to find UMVUE of ratio of $g(\theta)$ and $h(\theta)$ i.e
, $g(\theta)/h(\theta)$?
- 1
0
votes
0
answers
54
views
Using consistency to prove unbiasedness of linear regression
I am not sure if this question has already been asked, but basically in an interview I was told that using consistency, we have that
$$E(\frac{Cov(X,Y)}{Var(X)}) = \frac{E(Cov(X,Y))}{E(Var(X))}$$
...
2
votes
1
answer
78
views
Is asymptotic unbiasedness different from unbiasedness in practice?
Given some estimator T for a parameter θ, by definition T is unbiased if its bias B(T) is 0. It is asymptotically unbiased if B(T) is not 0, but some value that tends to 0 as n goes to infinity. My ...
4
votes
1
answer
91
views
Unbiased estimator of $2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$
For a random variable $x$, how would I go about creating an unbiased estimator of the following quantity?
$$R=2\left(\frac{E[x^2]}{E[x]^2}-1\right)^{-1}$$
For instance, when $x$ comes from from chi-...
- 5,859
0
votes
1
answer
71
views
Confusion regarding proof that the variance estimator is unbiased for finite population
Going through Sharon L. Lohr's Sampling design book (2nd Edition), I have no issues with the content all the way until it goes into the proof in chapter 2 on SRSWOR that $E[s^2] = S^2$, where $S^2$ is ...
- 103
1
vote
0
answers
129
views
Linear regression on mean + standard deviation
I have observations made at different times from a normally distributed real random variable whose mean and standard deviation both vary linearly with time. How can I estimate these two linear ...
- 111
2
votes
1
answer
129
views
why not using sample variance (instead of MSE) to estimate the error variance in linear regression?
Assuming the true equation for Y is linear as below:
$$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$
Assuming X is fixed, then the variance of each Y is:
$$var(Y_i )=var(\epsilon_i)=\sigma^2$$
In order to ...
- 309
4
votes
1
answer
185
views
Unbiased estimator for $\mu_1/\mu_2$
Let $X_1,X_2,\ldots,X_n$ and $Y_1,Y_2,\ldots,Y_n$ be independent random samples from $N(\mu_1,1)$ and $N(\mu_2,1)$ populations respectively with $\mu_2\neq0$.
I need to find an unbiased estimator for $...
- 43
4
votes
0
answers
135
views
Unbiasing estimator of $\|\Sigma\|_F^2$
I have access to samples of some distribution with second-moment matrix $\Sigma=E[xx^T]$ and need an estimate of $\|\Sigma\|_F^2$ (which can be used to set optimal size for LMS)
We can use Frobenius ...
- 5,859
0
votes
0
answers
29
views
Unbiasedness properties of Ratio/Proportion-type estimator
I have a ratio estimator, $\hat{a} = n_1/(n_0+n_1)$, where $n_x$ refers to the frequency of $x$-valued data. Note that, $E(n_0)$ and $E(n_1)$ exists and strictly positive.
Usually, to show an ...
- 364
4
votes
0
answers
44
views
How would you justify to a non-statistician why we should use an unbiased estimator instead of a maximum likelihood estimator?
Say we have the maximum likelihood estimator (which is usually biased) and an unbiased estimator and the sample size is small enough that these estimator are substantially different in magnitude. We'...
- 1,213
1
vote
0
answers
31
views
Optimal combination of correlated estimations
Consider two random unbiased estimates $\hat X_1,$, $\hat X_2$ of a parameter (complex number) $x$, with estimation errors $E_1 = \hat X_1-x$, $E_2 = \hat X_2-x$.
If the random variables $E_1$, $E_2$ ...
- 875