Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
745
questions
3
votes
1
answer
171
views
Conditions in law of large numbers
The (Strong) Law of large numbers states that
$ \frac{1}{N}\sum_{k=1}^N h(X_k) \rightarrow \mathbb{E}\left[h(X)\right]$
a.s in $\mu$ as $N\rightarrow \infty$.
but I can't find any conditions on $h(...
- 1,755
8
votes
3
answers
2k
views
Parameter estimation of exponential distribution with biased sampling
I want to calculate the parameter $\lambda$ of the exponential distribution $e^{-\lambda x}$ from a sample population taken out of this distribution under biased conditions. As far as I know, for a ...
- 81
2
votes
0
answers
97
views
Worst-case error related to Cramer-Rao bound
Asked this previously on Math.SE, maybe this fits here.
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of ...
- 121
2
votes
1
answer
194
views
Do we need an unbiased estimator of the variance?
"Although it is nice to have an unbiased estimator of the variance, we do not really need it to understand the relation between our independent variable and our dependent variable. Why?"
I think I ...
- 21
4
votes
2
answers
150
views
Is $s^2$ a good estimator?
I'm studying multivariate statistical analysis this semester.
In our text book, the author said that " A measure of spread is
provided by the sample variance, defined for $n$ measurements on the
...
- 635
6
votes
1
answer
247
views
Parameter estimation of a power spectrum equal to a power law + white noise
Given $X_t$ a multivariate random gaussian variable of covariance matrix $N_{tt'}$ diagonal in Fourier space (sampling is equally spaced),
I would like to parametrise its power spectrum as:
$S_X(f) = ...
- 61
3
votes
1
answer
1k
views
Asymptotic normality of MLE in exponential with higher-power x
Given the distribution:
$f(x;\theta) = \frac{3}{\theta}x^2e^{-x^3/\theta}$ if $x>0$
the MLE for $\theta$ is $\frac{1}{n}\sum_{i=1}^n x_i^3$. It's an unbiased estimator with variance $\theta^2/n$. ...
- 131
5
votes
2
answers
6k
views
Why doesn't the Cramér-Rao lower bound apply?
Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And
$f_\theta=0$ if $ x <...
0
votes
0
answers
3k
views
How to get unbiased estimate for error variance in ordinary linear model [duplicate]
I would like to estimate the variance of the error term in the ordinary linear regression model. The obvious estimate is the sample variance of the residuals, however this estimate consistently ...
- 163
4
votes
1
answer
349
views
How to handle sets with less than $k$ elements when using a single hash function to minhash?
A minhash implementation with multiple hash functions can easily handle comparisons between sets with a vastly different number of elements because the denominator of the unbiased estimator $k$ is ...
- 585
8
votes
1
answer
2k
views
Is this an unbiased estimator for standard deviation of normal distribution?
Suppose we have $n$ samples, with mean $\mu$.
Calculate the average absolute distance from $\mu$, i.e.,
$$
y = \frac{1}{n} \sum_{i=1}^n |X_i - \mu| \>.
$$
Then, take as an estimate of the ...
- 211
5
votes
1
answer
2k
views
Two unbiased estimators for the same quantity
In several situations, I have two unbiased estimators, and I know one of them is better (lower variance) than the other. However, I would like to get as much information as possible, and I would like ...
- 10.5k
2
votes
2
answers
3k
views
Does efficiency imply unbiased and consistency?
If I can prove that for an estimator $\hat{k}( \theta)$ I can write:
$$\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} = a(n, \theta)(\hat{\theta} - \theta)$$
Am i sure that the estimator is ...
- 756
193
votes
3
answers
233k
views
What is the difference between a consistent estimator and an unbiased estimator?
What is the difference between a consistent estimator and an unbiased estimator?
The precise technical definitions of these terms are fairly complicated, and it's difficult to get an intuitive feel ...
- 2,680
2
votes
1
answer
1k
views
Is a biased or unbiased estimator used for pooled SD in calculating Cohen's d?
When calculating Cohen's $d$ for independent samples, you must use a pooled $SD$. However, I have seen both of these:
$$SD_{\text{pooled1}} = \sqrt{\frac{ (n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2}}$...
- 93
21
votes
2
answers
2k
views
Is there an unbiased estimator of the Hellinger distance between two distributions?
In a setting where one observes $X_1,\ldots,X_n$ distributed from a distribution with density $f$, I wonder if there is an unbiased estimator (based on the $X_i$'s) of the Hellinger distance to ...
- 99.8k
5
votes
2
answers
1k
views
Can the ratio importance sampling estimate by made to be unbiased with resampling?
Consider approximating the following integral:
$$
\mathcal{Z} = \int h(x) \pi(x) dx
$$
Where $\pi$ is known only up to a normalizing constant, that is, $\pi(x) = \hat{\pi}(x)/\mathcal{Z}_\pi$. We can ...
- 1,218
3
votes
0
answers
291
views
A little confusion on Profile likelihood
As we all know, profile likelihood is an effective method for the estimation of conditional parametric model. But I still don't know exactly why it works. Profile likelihood was thoroughly studied by ...
- 643
0
votes
0
answers
295
views
Parameter estimation from a Normal distribution
Please can you check if am I correct?
I have a random variable $X$ normally distributed with mean $\mu$ and variance $\sigma^2$.
I generate two independent sample $T_1$ and $T_2$ with $T_1 < T_2$ ...
- 211
3
votes
0
answers
126
views
Positive estimator
Suppose that one can construct an unbiased estimator $X$ of the quantity $E$, is there a way of getting an unbiased and positive estimator of $E^2$? Indeed, if $X_1$ and $X_2$ are two independent ...
- 549
1
vote
1
answer
783
views
How to show unbiased estimator of combination of bernoulli and normal variables?
$X_1, X_2, \ldots, X_n$ is a random sample from $\mathrm{Bernoulli}(\theta)$, $\epsilon_1, \epsilon_2, \ldots, \epsilon_n$ are independent $\mathcal N(0, \sigma^2)$, independent of $X_i$.
Define $...
- 107
6
votes
1
answer
205
views
What is distribution of lengths of gaps between occurrences of ones in Bernoulli process?
Which distribution fits the following data? Data are generated by the process:
$X_t, \, t=\{1,2,3,\ldots,n\}$ is equal 1 with probability $p$, and 0 with probability $(1-p)$ for each $t$.
What is ...
- 1,667
8
votes
1
answer
2k
views
Sequential Monte Carlo (particle filter) with Metropolis-Hastings weighting
Let's say we are interested in approximating the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution known only up ...
- 1,218
6
votes
1
answer
1k
views
Finding a minimum variance unbiased (linear) estimator
Here is a basic question that perhaps has a simple answer, but one that I was not able to find by quickly scanning the literature.
Suppose that I have a collection of $n$ unopened boxes. Each box $i$...
- 238
10
votes
1
answer
1k
views
Optimal importance sampling with ratio estimator
We want to approximate the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution, also for simplicity, let's assume ...
- 1,218
3
votes
3
answers
278
views
Two questions on significance testing
Suppose you have a population and some measurement which you could do on each member of the population (e.g. the population could be all the people in the world, and the measurement could be height). ...
- 195
3
votes
1
answer
2k
views
Why is the design effect in most sample studies taken as 1.25?
Why is the design effect in most sample studies taken as 1.25? Who and how was it calculated first of all?
- 31
22
votes
4
answers
24k
views
What does "unbiasedness" mean?
What does it mean to say that "the variance is a biased estimator".
What does it mean to convert a biased estimate to an unbiased estimate through a simple formula. What does this conversion do ...
- 2,807
2
votes
1
answer
1k
views
Combining two unequal normal distributions
Let $X_1$ and $X_2$ be independent, normal distributed random variables with equal mean $\mu$ but non-equal standard deviations $\sigma_1$ and $\sigma_2$.
Suppose I know $\sigma_1$ and $\sigma_2$ and ...
- 267
7
votes
1
answer
2k
views
How to estimate the absolute expected difference?
Suppose we have two random variables $X$ and $Y$ with unknown distributions. I am looking for an unbiased estimator for the absolute expected difference:
$$
| E \{ X - Y \} | .
$$
For instance, ...
- 263
56
votes
2
answers
6k
views
Intuition behind why Stein's paradox only applies in dimensions $\ge 3$
Stein's Example shows that the maximum likelihood estimate of $n$ normally distributed variables with means $\mu_1,\ldots,\mu_n$ and variances $1$ is inadmissible (under a square loss function) iff $n\...
- 1,564
4
votes
2
answers
206
views
Bias in sampling for set intersections
Say I have 2 sets, $A$ and $B$ with $n_{A}$ and $n_{B}$ elements respectively, which I assume is known. I would like to estimate $| A \bigcup B |$ using samples of $\tilde{A} \subset A$ and $ \tilde{...
- 545
8
votes
3
answers
2k
views
When estimating variance, why do unbiased estimators divide by n-1 yet maximum likelihood estimates divide by n?
I am totally confused: On the one hand you can read all kinds of explanations why you have to divide by n-1 to get an unbiased estimator for the (unknown) population variance (degrees of freedom, not ...
- 6,066
13
votes
4
answers
1k
views
Unbiased estimator for the smaller of two random variables
Suppose $X \sim \mathcal{N}(\mu_x, \sigma^2_x)$ and $Y \sim \mathcal{N}(\mu_y, \sigma^2_y)$
I am interested in $z = \min(\mu_x, \mu_y)$. Is there an unbiased estimator for $z$?
The simple estimator ...
- 131
2
votes
2
answers
919
views
Seeing if estimators are unbiased
I have the pdf
$$f(y ; \theta) = \frac{1}{\theta} \exp( \frac{-y}{\theta}), \ y > 0$$
and I'm supposed to determine if the following two estimators are unbiased or not: $ \hat \theta = nY_{min} $...
- 499
3
votes
3
answers
3k
views
Analyzing the difference between two datasets where one is a subset of the other
I apologize in advance for the vague title, but I couldn't think of anything better.
I have two datasets, where one is a very small subset of the other. The percentage of people who have a specific ...
Sam Thropton
7
votes
1
answer
2k
views
Estimation of probability of a success in binomial distribution
Let's say we have two biased coins. The probability of tossing a head on the first coin is $\alpha$ and the probability of tossing a head on the second coin is $1-\alpha$. We toss both coins $n$ times ...
- 3,954
6
votes
2
answers
360
views
Is there a bias correction for effect size in a data mining context?
Given $K$ possible 'treatments' of some kind, and independent observations of some response under those treatments, say $X_{i,k}$ for $i=1,\ldots,n_k$ and $k=1,\ldots,K$, I am faced with the classical ...
- 14.2k
9
votes
2
answers
2k
views
Is sample kurtosis hopelessly biased?
I am looking at the sample kurtosis of a fairly skewed random variable, and the results seem inconsistent. To simply illustrate the problem, I looked at the sample kurtosis of a log-normal RV. In R (...
- 14.2k
14
votes
2
answers
1k
views
Model for population density estimation
A database of (population, area, shape) can be used to map population density by assigning a constant value of population/area to each shape (which is a polygon such as a Census block, tract, county, ...
- 309k
9
votes
2
answers
7k
views
Microsoft Excel formula for variance
According to Microsoft Excel Help:
VAR uses the following formula:
where x is the sample mean
AVERAGE(number1,number2,…) and n is
the sample size.
Shouldn't it be n, rather than n - 1, in the ...
- 827
20
votes
4
answers
3k
views
OLS is BLUE. But what if I don't care about unbiasedness and linearity?
The Gauss-Markov theorem tells us that the OLS estimator is the best linear unbiased estimator for the linear regression model.
But suppose I don't care about linearity and unbiasedness. Then is ...
25
votes
3
answers
3k
views
Unbiased estimation of covariance matrix for multiply censored data
Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
- 309k
5
votes
6
answers
930
views
Basic question regarding variance and stdev of a sample
Suppose there is a very big (infinite?) population of normally distributed values with unknown mean and variance.
Suppose also that we have a sample, S, of n values from the entire population. We can ...
- 113
18
votes
5
answers
1k
views
Why do US and UK Schools Teach Different methods of Calculating the Standard Deviation?
As I understand UK Schools teach that the Standard Deviation is found using:
whereas US Schools teach:
(at a basic level anyway).
This has caused a number of my students problems in the past as ...
- 526