Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...
3
votes
2answers
131 views
Sufficient statistics, MLE and unbiased estimators of uniform type distribution
Let $X_1, \dots, X_n$ denote a random sample of size n from the probability distribution with pdf:
$$ f_X(x|\theta_1, \theta_2) = \frac{1}{\theta_2 - \theta_1} \ I(x)_{[\theta_1,\theta_2]} \ ...
0
votes
0answers
30 views
Bias in EM estimation for a mixture of normal distributions
Are the parameter estimates for a mixture of two normal distribution using EM algorithm biased or unbiased?
More specifically, if I use the EM-algorithm to obtain ML estimates of $μ_1$, $μ_2$, ...
1
vote
1answer
14 views
Unbiased estimator from two SRS less duplicates
Suppose I take two independent random samples from a population of size $N$: the first is a simple random sample of size $n$ and the second is a simple random sample of size $m$. Let set $S$ contain ...
2
votes
1answer
347 views
How to show unbiased estimator of combination of bernoulli and normal variables?
$X_1, X_2, \ldots, X_n$ is a random sample from $\mathrm{Bernoulli}(\theta)$, $\epsilon_1, \epsilon_2, \ldots, \epsilon_n$ are independent $\mathcal N(0, \sigma^2)$, independent of $X_i$.
Define ...
1
vote
0answers
30 views
Unbiased estimate of the semi-partial correlation
Is the sample semi-partial correlation a biased estimate of the population semi-partial correlation?
If it is biased, what is an unbiased estimator of the population semi-partial correlation?
Are ...
-2
votes
0answers
55 views
Derive variance of the least squares estimator for $\beta$ [closed]
I want to prove the least squares estimator. Here is my proof:
$\beta = X'(X'X)^{-1} y$
$$Var((\hat \beta - E(\beta)(\hat \beta - E(\beta))'=E(\hat \beta - \beta)(\hat \beta - \beta)'$$
$(\hat ...
3
votes
1answer
48 views
Will two estimators converge to the same answer?
Say I have two estimators for the same quantity and using the same model, $E[f(X)]$. I also know that these two estimators are consistent, meaning, if we have a lot of data, they will be close to the ...
1
vote
1answer
128 views
Understanding “unbiased ridge regression”
Could someone please explain what "$J$" consists of in this paper, equation 1.5.
$$
J \sim N(\beta, \sigma^2 I/k)
$$
What's $\beta$ here? What's $N$?
Also, why are they putting that much effort in ...
2
votes
4answers
239 views
Why must one trade off between bias and variance?
Apparently, a learning algorithm must make a trade off between bias and variance when producing a hypothesis. Bias means systematic deviation from data. Variance refers to the error due to ...
2
votes
0answers
107 views
Determining variance of an U.M.V.U.E
Given $X_1...X_n$ i.i.d. Bernoulli R.V.s with parameter $\theta$, I found an UMVUE for $\tau(\theta)=\theta^2+\theta$ with Rao-Blackwellization. That process seemed fairly straight-forward to me, and ...
3
votes
0answers
88 views
Unbiased hypothesis tests
Is there some textbook or expository account showing that the definition of "unbiased test" bears the same sort of relation to "unbiased estimator" that interval estimation generally bears to ...
0
votes
0answers
31 views
counterexample for the sufficient condition required for consistency
We know that if an estimator is an unbiased estimator of theta and if its variance tends to 0 as n tends to infinity then it is a consistent estimator for theta. But this is a sufficient and not a ...
0
votes
0answers
113 views
Let X1,X2,…,Xn be i.i.d. N(θ1, θ2), please prove that E[(x1-θ1)^4] = 3θ2^2
If x$_{1}$, x$_{2}$,...,x$_{n}$ is sampled from N($\theta$$_{1}$, $\theta$$_{2}$), how can I prove that E [(x$_{1}$ - $\theta$$_{1}$)$^{4}$] = 3$\theta$$_{2}$$^{2}$?
I started off this question ...
0
votes
2answers
59 views
Bias in Estimators of Lognormal
I am modelling a process distributed as a 2 parameter lognormal distribution; determining the parameters by maximum likelihood.
I have simulated the bias in the estimators (logmean and logsd) as well ...
0
votes
0answers
59 views
Is there a closed form unbiased estimator of the median of n exponential variables from their mean? [closed]
Fix $n$. Suppose $T_1, \ldots, T_n \sim Exp(\lambda)$
For $n > 1$, is there ever an unbiased estimator of the median from a sample mean?
My attempt
$$\begin{align}P(median \le x) &= {K ...
13
votes
1answer
414 views
Is there an unbiased estimator of an Hellinger distance?
In a setting where one observes $X_1,\ldots,X_n$ distributed from a distribution with density $f$, I wonder if there is an unbiased estimator (based on the $X_i$'s) of the Hellinger distance to ...
8
votes
3answers
326 views
Parameter estimation of exponential distribution with biased sampling
I want to calculate the parameter $\lambda$ of the exponential distribution $e^{-\lambda x}$ from a sample population taken out of this distribution under biased conditions. As far as I know, for a ...
3
votes
1answer
115 views
What is the name of the estimator that takes the mean of likelihood?
Let $X,Y$ be input and output (observed) continuous variables in $\mathbb{R}$. Let $\{y_1,...,y_n\}$ be the set of $n$ observations. Is there a name for the estimator $\hat x = \int_{x \in X} x ...
1
vote
1answer
268 views
Show that $\hat\theta=\frac{2 \bar Y- 1}{1- \bar Y}$ is an unbiased estimator for $\theta$
Let $Y_1,Y_2,...,Y_n$ denote a random sample from the probability density function
$$f(y| \theta)= \begin{cases} ( \theta +1)y^{ \theta}, & 0 < y<1 , \theta> -1 \\ 0, & ...
1
vote
0answers
256 views
Finding MVUE for a Poisson distribution
The number of breakdowns Y per day for a certain machine is a Poisson random variable with mean $\lambda$. The daily cost of repairing these break downs is given by $C=3Y^2$ If $Y_1, Y_2, ..., Y_n$ ...
5
votes
1answer
175 views
What is the UMVUE for $\sigma^2$ in $\mathcal N(0, \sigma^2 )$?
By using the exponential class factorization theorem, I came up with $Y = \sum (x_i)^2$ to be the complete and sufficient statistics for $\sigma^2$ .
Using this sufficient statistic as a condition, ...
2
votes
1answer
115 views
Estimate the second moment of a latent variable using a conditionally unbiased proxy
The Setup: Let $X_t$ denote an unobservable stochastic sequence where the first two unconditional moments are finite constants; ie $\mathbb{E} X_t = \mu < \infty$ and $\mathbb{E} X_t^2 = \gamma ...
2
votes
1answer
113 views
Expectation of the sample median for symmetric distributions
Is the sample median an unbiased estimator of the population mean when the distribution is symmetric?
0
votes
0answers
61 views
How do I explain that software implemented model selection procedures should not be used unsupervised?
I know that people generally say that procedures which select a model based on information criterion lead to inconsistent model selections.
I read a paper by Leeb and Potscher (2005), MODEL SELECTION ...
7
votes
2answers
345 views
Estimating parameters of a normal distribution: median instead of mean?
The common approach for estimating the parameters of a normal distribution is to use the mean and the sample standard deviation / variance.
However, if there are some outliers, the median and the ...
7
votes
1answer
257 views
Bias correction in weighted variance
For unweighted variance
$$\text{Var}(X):=\frac{1}{n}\sum_i(x_i - \mu)^2$$
there exists the bias corrected sample variance, when the mean was estimated from the same data:
...
11
votes
3answers
4k views
What is the difference between a consistent estimator and an unbiased estimator?
I'm really surprised that nobody appears to have asked this already...
When discussing estimators, two terms frequently used are "consistent" and "unbiased". My question is simple: what's the ...
2
votes
2answers
462 views
Why doesn't the Cramer-Rao lower bound apply?
Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And
$f_\theta=0$ if $ x ...
9
votes
5answers
407 views
Why do US and UK Schools Teach Different methods of Calculating the Standard Deviation?
As I understand UK Schools teach that the Standard Deviation is found using:
whereas US Schools teach:
(at a basic level anyway).
This has caused a number of my students problems in the past as ...
4
votes
1answer
81 views
What kind of experiment might Hiawatha have designed?
I am trying to understand the poem Hiawatha Designs an Experiment because it looks like the kind of in-joke that it would be nice to get, as a statistician. Here is what I would like to know:
...
2
votes
1answer
61 views
Conditions in law of large numbers
The (Strong) Law of large numbers states that
$ \frac{1}{N}\sum_{k=1}^N h(X_k) \rightarrow \mathbb{E}\left[h(X)\right]$
a.s in $\mu$ as $N\rightarrow \infty$.
but I can't find any conditions on ...
0
votes
0answers
76 views
Unbiased estimate of variance
I'm trying to teach myself statistics. Please help me with this estimation problem:
Let 12,15,14,16,13 be real observations from $\mathcal{N}(\mu_1,σ_1^2)$ and 2,6,7 be real observations from ...
1
vote
0answers
39 views
Worst-case error related to Cramer-Rao bound
Asked this previously on Math.Stackexchange, maybe this fits here.
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple ...
2
votes
0answers
106 views
Do we need an unbiased estimator of the variance?
"Although it is nice to have an unbiased estimator of the variance, we do not really need it to understand the relation between our independent variable and our dependent variable. Why?"
I think I ...
9
votes
3answers
1k views
What does “unbiasedness” mean?
What does it mean to say that "the variance is a biased estimator".
What does it mean to convert a biased estimate to an unbiased estimate through a simple formula. What does this conversion do ...
3
votes
2answers
100 views
Is $s^2$ a good estimator?
I'm studying multivariate statistical analysis this semester.
In our text book, the author said that " A measure of spread is
provided by the sample variance, defined for $n$ measurements on ...
3
votes
0answers
66 views
Parameter estimation of a power spectrum equal to a power law + white noise
Given $X_t$ a multivariate random gaussian variable of covariance matrix $N_{tt'}$ diagonal in Fourier space (sampling is equally spaced),
I would like to parametrise its power spectrum as:
$S_X(f) = ...
2
votes
6answers
613 views
Basic question regarding variance and stdev of a sample
Suppose there is a very big (infinite?) population of normally distributed values with unknown mean and variance.
Suppose also that we have a sample, S, of n values from the entire population. We can ...
1
vote
0answers
160 views
Sufficient, Complete Sufficient, UMVUE, Rao-Blackwell, Admissible. What are ties between these?
I am taking stat inference course. I have some trouble understanding some these terms:
Sufficient Statistics: a stat that does not depend on the parameter, say $\Sigma X$ for normal distribution
...
13
votes
3answers
729 views
Unbiased estimation of covariance matrix for multiply censored data
Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
2
votes
0answers
135 views
Asymptotic normality of MLE in exponential with higher-power x
Given the distribution:
$f(x;\theta) = \frac{3}{\theta}x^2e^{-x^3/\theta}$ if $x>0$
the MLE for $\theta$ is $\frac{1}{n}\sum_{i=1}^n x_i^3$. It's an unbiased estimator with variance $\theta^2/n$. ...
2
votes
1answer
100 views
How to handle sets with less than $k$ elements when using a single hash function to minhash?
A minhash implementation with multiple hash functions can easily handle comparisons between sets with a vastly different number of elements because the denominator of the unbiased estimator $k$ is ...
5
votes
1answer
283 views
Finding a minimum variance unbiased (linear) estimator
Here is a basic question that perhaps has a simple answer, but one that I was not able to find by quickly scanning the literature.
Suppose that I have a collection of $n$ unopened boxes. Each box ...
3
votes
0answers
254 views
Is this an unbiased estimator for standard deviation of normal distribution?
Suppose we have $n$ samples, with mean $\mu$.
Calculate the average absolute distance from $\mu$, i.e.,
$$
y = \frac{1}{n} \sum_{i=1}^n |X_i - \mu| \>.
$$
Then, take as an estimate of the ...
4
votes
1answer
180 views
Two unbiased estimators for the same quantity
In several situations, I have two unbiased estimators, and I know one of them is better (lower variance) than the other. However, I would like to get as much information as possible, and I would like ...
1
vote
1answer
276 views
Does efficiency imply unbiased and consistency?
If I can prove that for an estimator $\hat{k}( \theta)$ I can write:
$$\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} = a(n, \theta)(\hat{\theta} - \theta)$$
Am i sure that the estimator is ...
2
votes
1answer
243 views
Is a biased or unbiased estimator used for pooled SD in calculating Cohen's d?
When calculating Cohen's $d$ for independent samples, you must use a pooled $SD$. However, I have seen both of these:
$$SD_{\text{pooled1}} = \sqrt{\frac{ (n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + ...
2
votes
2answers
135 views
Can the ratio importance sampling estimate by made to be unbiased with resampling?
Consider approximating the following integral:
$$
\mathcal{Z} = \int h(x) \pi(x) dx
$$
Where $\pi$ is known only up to a normalizing constant, that is, $\pi(x) = \hat{\pi}(x)/\mathcal{Z}_\pi$. We can ...
1
vote
0answers
101 views
A little confusion on Profile likelihood
As we all know, profile likelihood is an effective method for the estimation of conditional parametric model. But I still don't know exactly why it works. Profile likelihood was thoroughly studied by ...
0
votes
0answers
129 views
Parameter estimation from a Normal distribution
Please can you check if am I correct?
I have a random variable $X$ normally distributed with mean $\mu$ and variance $\sigma^2$.
I generate two independent sample $T_1$ and $T_2$ with $T_1 < T_2$ ...
