Hot answers tagged unbiased-estimator
13
The first formula is the population standard deviation and the second formula is the the sample standard deviation. The second formula is also related to the unbiased estimator of the variance - see wikipedia for further details.
I suppose (here) in the UK they don't make the distinction between sample and population at high school. They certainly don't ...
13
The dichotomy between the cases $d < 3$ and $d \geq 3$ for the admissibility of the MLE of the mean of a $d$-dimensional multivariate normal random variable is certainly shocking.
There is another very famous example in probability and statistics in which there is a dichotomy between the $d < 3$ and $d \geq 3$ cases. This is the recurrence of a simple ...
13
To define the two terms without using too much technical language:
An estimator is consistent if, as the sample size increases, the estimator "converges" to the true value of the parameter being estimated. To be slightly more precise - consistency means that, as the sample size increases, the sampling distribution of the estimator becomes increasingly ...
12
You can find everything here. However, here is a brief answer.
Let $\mu$ and $\sigma^2$ be the mean and the variance of interest; you wish to estimate $\sigma^2$ based on a sample of size $n$.
Now, let us say you use the following estimator:
$S^2 = \frac{1}{n} \sum_{i=1}^n (X_{i} - \bar{X})^2$,
where $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ is the ...
12
You aren't strictly taking the "mean" of the likelihood, because the Likelihood function isn't a probability distribution over x. It isn't even a probability distribution anyway, but assuming you have a likelihood function that you can normalize into a PDF then it would be the probability of $Y$ not of $X$. This is a likelihood weighted average of $X$.
I ...
10
Unbiased estimates are typical in introductory statistics courses because they are: 1) classic, 2) easy to analyze mathematically. The Cramer-Rao lower bound is one of the main tools for 2). Away from unbiased estimates there is possible improvement. The bias-variance trade off is an important concept in statistics for understanding how biased estimates ...
8
The maximum likelihood estimator for the parameter of the exponential distribution under type II censoring can be derived as follows. I assume the sample size is $m$, of which the $n < m$ smallest are observed and the $m - n$ largest are unobserved (but known to exist.)
Let us assume (for notational simplicity) that the observed $x_i$ are ordered: $0 ...
7
This is just a couple of comments not an answer (don't have enough rep. point).
(1). There is an explicit formula for the bias of the simple estimator $min(\bar{x},\bar{y})$ here:
Clark, C. E. 1961, Mar-Apr. The greatest of a finite set of random variables.
Operations Research 9 (2): 145–162.
Not sure how this helps though
(2). This is just intuition, ...
7
I don't know if you are OK with the Bayes Estimate? If yes, then depending on the Loss function you can obtain different Bayes Estimates. A theorem by Blackwell states that Bayes Estimates are never unbiased. A decision theoretic argument states that every admissible rule ((i.e. or every other rule against which it is compared, there is a value of the ...
7
I don't know how to construct (if it exists) an unbiased estimator of the Hellinger distance. It seems possible to construct a consistent estimator. We have some fixed known density $f_0$, and a random sample $X_1,\dots,X_n$ from a density $f>0$. We want to estimate
$$
H(f,f_0) = \sqrt{1 - \int_\mathscr{X} \sqrt{f(x)f_0(x)}\,dx} = \sqrt{1 - ...
6
Because nobody has yet answered the final question--namely, to quantify the differences between the two formulas--let's take care of that.
For many reasons, it is appropriate to compare standard deviations in terms of their ratios rather than their differences. The ratio is
$$s_n / s = \sqrt{\frac{N-1}{N}} = \sqrt{1 - \frac{1}{N}} \approx 1 - ...
6
The estimator is biased, regardless.
Note first that $\alpha$ is not identifiable because you cannot distinguish between $\alpha$ and $1-\alpha$. Let's accommodate this problem by allowing that we don't care which coin is which and stipulating (arbitrarily, but with no loss of generality), that $0 \le \alpha \le 1/2$.
It's reasonable, and conventional, ...
6
First of all, in your example involving data drawn from a contaminated Gaussian distribution, you'd get better results if you had used the $\text{mad}$ instead of $\text{med}|x-\text{med}(x)|$ where $\text{mad}(x)$ is:
$$\text{mad}=1.4826\times\text{med}|x-\text{med}(x)|$$
(by the way you gotta be careful here: non-statisticians and many softwares often ...
6
This answer cannot be correct. An estimator cannot depend on the values of the parameters: since they are unknown it would mean that you cannot compute the estimate.
An unbiased estimator of the variance for every distribution (with finite second moment) is
$$ S^2 = \frac{1}{n-1}\sum_{i=1}^n (y_i - \bar{y})^2.$$
By expanding the square and using the ...
5
This is analogous to fixed-effect meta-analysis. The best (minimum-variance unbiased) estimator would be the inverse-variance weighted mean
$$\hat{\mu} = \frac{ \frac{1}{\sigma_1^2} \sum x_{1i} + \frac{1}{\sigma_2^2} \sum x_{2i} }{n/\sigma_1^2 + m/\sigma_2^2}$$
When $n=m=1$ that reduces to
$$\hat{\mu} = \frac{ x_{11} / \sigma_1^2 + x_{21} / \sigma_2^2 ...
5
Make a table:
No renewal Renewal Total
---------- ------- ------
Attribute No 79800 200 80000
Yes 19700 300 20000
---------------------------------------
Total 99500 500 100000
The computations are:
Number of non-renewers = 100,000 - 500 = 99,500.
"Attribute" is 20% of all ...
5
There's a bias correction. It's not huge. I believe the sampling variance of the kurtosis is proportional to the eighth (!) central moment, which can be enormous for a lognormal distribution. You would need millions of trials (or far more) in a simulation to detect bias unless the CV is tiny. (Plot a histogram of kvals to see how extraordinarily skewed ...
5
This is an interesting [and very far from "stupid"] question that actually bothered me for a while! We cover it in Monte Carlo Statistical Methods (Section 3.3.2, pages 95-96). The crux of it is that, by dividing by the sum of the weights the optimality vanishes. It is actually easy to see when $h$ is a positive function. In this case,
$$
w(x) h(x) = 1
$$
...
5
The time you have to wait till the next one is a geometric variable $X\sim\mathcal{G}(p)$ with probability parameter $p$, i.e.
$$
\mathbb{P}(X=k) = (1-p)^k p \quad k=0,1,2,\ldots
$$
Fitting your distribution to the data presumably means estimating $p$ by $\hat p$ and using the pluggin distribution $\mathcal{G}(\hat p)$ for all purposes. If you do not want ...
5
Consistency of an estimator means that as the sample size gets large the estimate gets closer and closer to the true value of the parameter. Unbiasedness is a finite sample property that is not affected by increasing sample size. An estimate is unbiased if its expected value equals the true parameter value. This will be true for all sample sizes and is ...
5
It results as a decomposition of the error function in two terms, representing "two opposing forces", in the sense that in order to reduce the bias error, you need your model to consider more possibilities to fit the data. But this on the other side increases the variance error. Also, the other way around: if your model fits too much (starts to fit noise, ...
5
@gui11aume is right of course. An outline of a derivation specific to a $\operatorname{Bin}(1,\pi)$ distribution follows:
Find the variance in terms of $\pi$ to reparameterize the probability mass function: $$\theta=\operatorname{Var}{Y_i}=\pi(1-\pi)$$
Find the maximum-likelihood estimator of $\theta$: ...
4
The second question seems to ask for a prediction interval for one future observation. Such an interval is readily calculated under the assumptions that (a) the future observation is from the same distribution and (b) is independent of the previous sample. When the underlying distribution is Normal, we just have to erect an interval around the difference ...
4
[Just on the R Style - @whuber has answered the Kurtsosis Q]
This was a bit too complicated to stick into a comment. For such simple loops like the one you use, we can combine @whuber's suggestion of encapsulating the simulation in a function with the replicate() function. replicate() takes care of allocation and running the loop for you. An example is ...
4
You are right that an unbiased estimator doesn't exist. The problem is that the parameter of interest is not a smooth function of the underlying data distribution due to non-differentiability at $\mu_x=\mu_y$.
The proof is as follows. Let $T(X,Y)$ be an unbiased estimator. Then $E_{\mu_x,\mu_y}[T(X,Y)]=\min\{\mu_x,\mu_y\}$. The left-hand side is ...
4
I'm still not quite sure how part (a) is different from part (b) but, from your comment above it appears you are now only asking about part (b), so:
If $\psi[\hat{\theta};(X,Y)] = 0$, then
$$
\sum_{i =1}^n(Y_i - \hat{\theta} X_i) \> = 0
$$
So,
$$ \sum_{i=1}^{n} Y_i = \hat{\theta} \cdot \sum_{i=1}^{n} X_i $$
Therefore $\hat{\theta} = ...
4
If you have two unbiased estimators $L_1$ and $L_2$ then accuracy is measured by variance (for a mean square error loss function).
Take $L=aL_1+(1-a)L_2$, a weighted average of the two with $0 < a < 1$.
$$\newcommand{\Var}{\mathrm{Var}}\newcommand{\Cov}{\mathrm{Cov}}
\Var(L) =a^2\Var(L_1) +(1-a)^2\Var(L_2) +2a(1-a) \Cov(L_1, L_2) \>.
$$
When ...
4
Given the answers and comments on both sites, you're probably done with your homework. The following is just a clarification for future visitors.
This is your statistical model: you have random variables $Y_1,\dots,Y_n$, which are independent and identically distributed, with $Y_i\sim\mathrm{Beta}(\theta +1,1)$, for $\theta>-1$.
An unbiased estimator ...
3
This response clarifies ocram's answer. The key reason (and common misunderstanding) for $E[S^2] \neq \sigma^2$ is that $S^2$ uses the estimate $\bar{X}$ which is itself estimated from data.
If you work through the derivation, you will see that the variance of this estimate $E[(\bar{X}-\mu)^2]$ is exactly what gives the additional $-\frac{\sigma^2}{n}$ ...
3
The explanation that @Ocram gave is great. To explain what he said in words: if we calculate $s^2$ by dividing just by $n$, (which is intuitive) our estimation of $s^2$ will be an underestimate. To compensate, we divide by $n-1$.
Here's an exercise: Make up a discrete probability with 2 outcomes, say $P(2) = .25$ and $P(6) = .75$. Find $\mu$ and $\sigma$ ...
Only top voted, non community-wiki answers of a minimum length are eligible
