Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an ...
0
votes
0answers
31 views
Bias in EM estimation for a mixture of normal distributions
Are the parameter estimates for a mixture of two normal distribution using EM algorithm biased or unbiased?
More specifically, if I use the EM-algorithm to obtain ML estimates of $μ_1$, $μ_2$, ...
1
vote
0answers
30 views
Unbiased estimate of the semi-partial correlation
Is the sample semi-partial correlation a biased estimate of the population semi-partial correlation?
If it is biased, what is an unbiased estimator of the population semi-partial correlation?
Are ...
3
votes
1answer
50 views
Will two estimators converge to the same answer?
Say I have two estimators for the same quantity and using the same model, $E[f(X)]$. I also know that these two estimators are consistent, meaning, if we have a lot of data, they will be close to the ...
3
votes
0answers
89 views
Unbiased hypothesis tests
Is there some textbook or expository account showing that the definition of "unbiased test" bears the same sort of relation to "unbiased estimator" that interval estimation generally bears to ...
2
votes
0answers
107 views
Determining variance of an U.M.V.U.E
Given $X_1...X_n$ i.i.d. Bernoulli R.V.s with parameter $\theta$, I found an UMVUE for $\tau(\theta)=\theta^2+\theta$ with Rao-Blackwellization. That process seemed fairly straight-forward to me, and ...
0
votes
0answers
31 views
counterexample for the sufficient condition required for consistency
We know that if an estimator is an unbiased estimator of theta and if its variance tends to 0 as n tends to infinity then it is a consistent estimator for theta. But this is a sufficient and not a ...
2
votes
4answers
242 views
Why must one trade off between bias and variance?
Apparently, a learning algorithm must make a trade off between bias and variance when producing a hypothesis. Bias means systematic deviation from data. Variance refers to the error due to ...
0
votes
0answers
113 views
Let X1,X2,…,Xn be i.i.d. N(θ1, θ2), please prove that E[(x1-θ1)^4] = 3θ2^2
If x$_{1}$, x$_{2}$,...,x$_{n}$ is sampled from N($\theta$$_{1}$, $\theta$$_{2}$), how can I prove that E [(x$_{1}$ - $\theta$$_{1}$)$^{4}$] = 3$\theta$$_{2}$$^{2}$?
I started off this question ...
0
votes
2answers
60 views
Bias in Estimators of Lognormal
I am modelling a process distributed as a 2 parameter lognormal distribution; determining the parameters by maximum likelihood.
I have simulated the bias in the estimators (logmean and logsd) as well ...
3
votes
2answers
131 views
Sufficient statistics, MLE and unbiased estimators of uniform type distribution
Let $X_1, \dots, X_n$ denote a random sample of size n from the probability distribution with pdf:
$$ f_X(x|\theta_1, \theta_2) = \frac{1}{\theta_2 - \theta_1} \ I(x)_{[\theta_1,\theta_2]} \ ...
0
votes
0answers
59 views
Is there a closed form unbiased estimator of the median of n exponential variables from their mean? [closed]
Fix $n$. Suppose $T_1, \ldots, T_n \sim Exp(\lambda)$
For $n > 1$, is there ever an unbiased estimator of the median from a sample mean?
My attempt
$$\begin{align}P(median \le x) &= {K ...
1
vote
1answer
16 views
Unbiased estimator from two SRS less duplicates
Suppose I take two independent random samples from a population of size $N$: the first is a simple random sample of size $n$ and the second is a simple random sample of size $m$. Let set $S$ contain ...
3
votes
1answer
115 views
What is the name of the estimator that takes the mean of likelihood?
Let $X,Y$ be input and output (observed) continuous variables in $\mathbb{R}$. Let $\{y_1,...,y_n\}$ be the set of $n$ observations. Is there a name for the estimator $\hat x = \int_{x \in X} x ...
1
vote
1answer
268 views
Show that $\hat\theta=\frac{2 \bar Y- 1}{1- \bar Y}$ is an unbiased estimator for $\theta$
Let $Y_1,Y_2,...,Y_n$ denote a random sample from the probability density function
$$f(y| \theta)= \begin{cases} ( \theta +1)y^{ \theta}, & 0 < y<1 , \theta> -1 \\ 0, & ...
1
vote
0answers
258 views
Finding MVUE for a Poisson distribution
The number of breakdowns Y per day for a certain machine is a Poisson random variable with mean $\lambda$. The daily cost of repairing these break downs is given by $C=3Y^2$ If $Y_1, Y_2, ..., Y_n$ ...
5
votes
1answer
175 views
What is the UMVUE for $\sigma^2$ in $\mathcal N(0, \sigma^2 )$?
By using the exponential class factorization theorem, I came up with $Y = \sum (x_i)^2$ to be the complete and sufficient statistics for $\sigma^2$ .
Using this sufficient statistic as a condition, ...
2
votes
1answer
116 views
Expectation of the sample median for symmetric distributions
Is the sample median an unbiased estimator of the population mean when the distribution is symmetric?
0
votes
0answers
61 views
How do I explain that software implemented model selection procedures should not be used unsupervised?
I know that people generally say that procedures which select a model based on information criterion lead to inconsistent model selections.
I read a paper by Leeb and Potscher (2005), MODEL SELECTION ...
2
votes
1answer
115 views
Estimate the second moment of a latent variable using a conditionally unbiased proxy
The Setup: Let $X_t$ denote an unobservable stochastic sequence where the first two unconditional moments are finite constants; ie $\mathbb{E} X_t = \mu < \infty$ and $\mathbb{E} X_t^2 = \gamma ...
7
votes
2answers
347 views
Estimating parameters of a normal distribution: median instead of mean?
The common approach for estimating the parameters of a normal distribution is to use the mean and the sample standard deviation / variance.
However, if there are some outliers, the median and the ...
1
vote
1answer
135 views
Understanding “unbiased ridge regression”
Could someone please explain what "$J$" consists of in this paper, equation 1.5.
$$
J \sim N(\beta, \sigma^2 I/k)
$$
What's $\beta$ here? What's $N$?
Also, why are they putting that much effort in ...
7
votes
1answer
263 views
Bias correction in weighted variance
For unweighted variance
$$\text{Var}(X):=\frac{1}{n}\sum_i(x_i - \mu)^2$$
there exists the bias corrected sample variance, when the mean was estimated from the same data:
...
4
votes
1answer
81 views
What kind of experiment might Hiawatha have designed?
I am trying to understand the poem Hiawatha Designs an Experiment because it looks like the kind of in-joke that it would be nice to get, as a statistician. Here is what I would like to know:
...
2
votes
1answer
61 views
Conditions in law of large numbers
The (Strong) Law of large numbers states that
$ \frac{1}{N}\sum_{k=1}^N h(X_k) \rightarrow \mathbb{E}\left[h(X)\right]$
a.s in $\mu$ as $N\rightarrow \infty$.
but I can't find any conditions on ...
0
votes
0answers
76 views
Unbiased estimate of variance
I'm trying to teach myself statistics. Please help me with this estimation problem:
Let 12,15,14,16,13 be real observations from $\mathcal{N}(\mu_1,σ_1^2)$ and 2,6,7 be real observations from ...
8
votes
3answers
327 views
Parameter estimation of exponential distribution with biased sampling
I want to calculate the parameter $\lambda$ of the exponential distribution $e^{-\lambda x}$ from a sample population taken out of this distribution under biased conditions. As far as I know, for a ...
1
vote
0answers
40 views
Worst-case error related to Cramer-Rao bound
Asked this previously on Math.Stackexchange, maybe this fits here.
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple ...
2
votes
0answers
106 views
Do we need an unbiased estimator of the variance?
"Although it is nice to have an unbiased estimator of the variance, we do not really need it to understand the relation between our independent variable and our dependent variable. Why?"
I think I ...
3
votes
2answers
100 views
Is $s^2$ a good estimator?
I'm studying multivariate statistical analysis this semester.
In our text book, the author said that " A measure of spread is
provided by the sample variance, defined for $n$ measurements on ...
3
votes
0answers
66 views
Parameter estimation of a power spectrum equal to a power law + white noise
Given $X_t$ a multivariate random gaussian variable of covariance matrix $N_{tt'}$ diagonal in Fourier space (sampling is equally spaced),
I would like to parametrise its power spectrum as:
$S_X(f) = ...
1
vote
0answers
160 views
Sufficient, Complete Sufficient, UMVUE, Rao-Blackwell, Admissible. What are ties between these?
I am taking stat inference course. I have some trouble understanding some these terms:
Sufficient Statistics: a stat that does not depend on the parameter, say $\Sigma X$ for normal distribution
...
2
votes
0answers
135 views
Asymptotic normality of MLE in exponential with higher-power x
Given the distribution:
$f(x;\theta) = \frac{3}{\theta}x^2e^{-x^3/\theta}$ if $x>0$
the MLE for $\theta$ is $\frac{1}{n}\sum_{i=1}^n x_i^3$. It's an unbiased estimator with variance $\theta^2/n$. ...
2
votes
2answers
462 views
Why doesn't the Cramer-Rao lower bound apply?
Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And
$f_\theta=0$ if $ x ...
2
votes
1answer
102 views
How to handle sets with less than $k$ elements when using a single hash function to minhash?
A minhash implementation with multiple hash functions can easily handle comparisons between sets with a vastly different number of elements because the denominator of the unbiased estimator $k$ is ...
3
votes
0answers
254 views
Is this an unbiased estimator for standard deviation of normal distribution?
Suppose we have $n$ samples, with mean $\mu$.
Calculate the average absolute distance from $\mu$, i.e.,
$$
y = \frac{1}{n} \sum_{i=1}^n |X_i - \mu| \>.
$$
Then, take as an estimate of the ...
4
votes
1answer
181 views
Two unbiased estimators for the same quantity
In several situations, I have two unbiased estimators, and I know one of them is better (lower variance) than the other. However, I would like to get as much information as possible, and I would like ...
1
vote
1answer
280 views
Does efficiency imply unbiased and consistency?
If I can prove that for an estimator $\hat{k}( \theta)$ I can write:
$$\frac{\partial l(X_1, \dots , X_n)}{\partial \theta} = a(n, \theta)(\hat{\theta} - \theta)$$
Am i sure that the estimator is ...
11
votes
3answers
4k views
What is the difference between a consistent estimator and an unbiased estimator?
I'm really surprised that nobody appears to have asked this already...
When discussing estimators, two terms frequently used are "consistent" and "unbiased". My question is simple: what's the ...
2
votes
1answer
243 views
Is a biased or unbiased estimator used for pooled SD in calculating Cohen's d?
When calculating Cohen's $d$ for independent samples, you must use a pooled $SD$. However, I have seen both of these:
$$SD_{\text{pooled1}} = \sqrt{\frac{ (n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + ...
13
votes
1answer
415 views
Is there an unbiased estimator of an Hellinger distance?
In a setting where one observes $X_1,\ldots,X_n$ distributed from a distribution with density $f$, I wonder if there is an unbiased estimator (based on the $X_i$'s) of the Hellinger distance to ...
2
votes
2answers
135 views
Can the ratio importance sampling estimate by made to be unbiased with resampling?
Consider approximating the following integral:
$$
\mathcal{Z} = \int h(x) \pi(x) dx
$$
Where $\pi$ is known only up to a normalizing constant, that is, $\pi(x) = \hat{\pi}(x)/\mathcal{Z}_\pi$. We can ...
1
vote
0answers
101 views
A little confusion on Profile likelihood
As we all know, profile likelihood is an effective method for the estimation of conditional parametric model. But I still don't know exactly why it works. Profile likelihood was thoroughly studied by ...
0
votes
0answers
129 views
Parameter estimation from a Normal distribution
Please can you check if am I correct?
I have a random variable $X$ normally distributed with mean $\mu$ and variance $\sigma^2$.
I generate two independent sample $T_1$ and $T_2$ with $T_1 < T_2$ ...
2
votes
0answers
81 views
Positive estimator
Suppose that one can construct an unbiased estimator $X$ of the quantity $E$, is there a way of getting an unbiased and positive estimator of $E^2$? Indeed, if $X_1$ and $X_2$ are two independent ...
2
votes
1answer
347 views
How to show unbiased estimator of combination of bernoulli and normal variables?
$X_1, X_2, \ldots, X_n$ is a random sample from $\mathrm{Bernoulli}(\theta)$, $\epsilon_1, \epsilon_2, \ldots, \epsilon_n$ are independent $\mathcal N(0, \sigma^2)$, independent of $X_i$.
Define ...
2
votes
1answer
89 views
What is distribution of lengths of gaps between occurrences of ones in Bernoulli process?
Which distribution fits the following data? Data are generated by the process:
$X_t, \, t=1,2,3,\ldots,n$ is equal 1 with probability $p$ and 0 with probability $(1-p)$ for each $t$.
What is the ...
4
votes
1answer
392 views
Sequential Monte Carlo (particle filter) with Metropolis-Hastings weighting
Let's say we are interested in approximating the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution known only up ...
5
votes
1answer
283 views
Finding a minimum variance unbiased (linear) estimator
Here is a basic question that perhaps has a simple answer, but one that I was not able to find by quickly scanning the literature.
Suppose that I have a collection of $n$ unopened boxes. Each box ...
3
votes
1answer
153 views
Optimal importance sampling with ratio estimator
This is probably a stupid question, but here goes: so we want to approximate the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is ...
2
votes
3answers
198 views
Two questions on significance testing
Suppose you have a population and some measurement which you could do on each member of the population (e.g. the population could be all the people in the world, and the measurement could be height). ...
