Questions tagged [rao-blackwell]
The Rao-Blackwell Theorem is a result which makes possible to better an estimator by conditioning on a sufficient statistic, preserving the expectation of the estimator.
54
questions
2
votes
2
answers
111
views
How does Rao-Blackwellization of the Metropolis-Hastings algorithm work?
I've read the paper A vanilla Rao-Blackwellization of Metropolis-Hastings algorithms, but I don't get what their actually suggested estimator is.
To give some detail, we are considering the following ...
- 71
0
votes
1
answer
110
views
Poisson: finding UMVUE for $\lambda + \lambda^2$ [closed]
Let $X_1,..., X_n$ be iid sample from the Poisson distribution with parameter $\lambda$. Find the UMVUE of $\lambda + \lambda^2$.
I know $T := \sum\limits_{i=1}^n X_i$ is complete and sufficient for $\...
0
votes
0
answers
270
views
What is the difference between MVB UMVUE and MVUE.?
Cramer Rao inequality gives MVB and if MVB exist it is MLE.
Rao Blackwell gives UMVUE, but isn’t when we have MVB estimator for unbiased it is UMVUE?
Then what is MVUE?
MVB minimum variance bound
...
- 13
4
votes
1
answer
178
views
Rao-Blackwellisation using non-sufficient statistics
The following is given as a remark in chapter 7 of Introduction to mathematical statistics Hogg and Craig, 8th edition. (It is mentioned as "Remark 7.3.1")
Now, I do understand that the ...
- 226
1
vote
3
answers
446
views
Rao-Blackwell and unbiased estimators of zero
In Casella & Berger we are trying to prove that any estimator $\phi$ based on a complete sufficient statistic T is the unique best unbiased estimator of its expectation.
However, in the preceding ...
- 1,086
1
vote
0
answers
108
views
Conditional Density Function on Underlying Exponential
Problem Statement: Suppose $Y_1, Y_2,\dots, Y_n$ are a random sample from an exponential distribution with mean $\theta.$ Let $\displaystyle U=\sum_{i=1}^n Y_i.$ Find the conditional density function ...
- 5,780
4
votes
1
answer
90
views
Rao-Blackwellization in Black Box VI
In the paper, "Black Box Variational Inference," by Ranganath et al. (2013), the authors derive a Rao-Blackwellized estimator of the gradient of the evidence lower bound with respect to a ...
- 41
3
votes
1
answer
161
views
Rao–Blackwellization of Metropolis–Hastings
I am trying to achieve a Rao–Blackwellization of Metropolis–Hastings algorithm. In the paper by Robert et al. 2018, the following is given.
\begin{align}
ℑ=&\frac{1}{T}\sum_{t=1}^Th(\theta^{(t)})=\...
- 143
6
votes
1
answer
435
views
Trying to make sense of claims regarding Rao-Blackwell and Lehmann-Scheffé for sufficient/complete statistics
I am currently trying to learn the two related concepts of the Rao-Blackwell theorem and the Lehmann-Scheffé theorem.
Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\...
- 1,436
3
votes
1
answer
855
views
Understanding the Rao-Blackwell Theorem
I've been reading up a lot on the practical applications of the Rao-Blackwell theorem. I do understand how the Bias and Variance and MSE aspects of the theorem fall in place (i.e. the mathematical ...
- 55
0
votes
1
answer
201
views
Finding UMVUE of function of poisson parameter
I am to estimate $\exp(-\lambda)\lambda^2/2$ from the distribution $Exp(\lambda) \sim \frac{e^{-\lambda}\lambda^x}{x!}$
I used the indicator function $W=\mathbb I_{2}(X_1)$ as an initial unbiased ...
- 123
2
votes
1
answer
1k
views
Rao Blackwell theorem on Bernoulli distribution
I need help with the following Problem: Let $X_1,...,X_n$ be a random sample of iid random variables,
$X_i\sim Ber(p), p\in (0,1)$. We want to estimate $\theta = p^2$. It is known, that $\hat{\theta}(...
- 41
1
vote
1
answer
219
views
Cramer-Rao Lower Bound Proof (fuzzy step)
The following is the derivation of the Cramer-Rao lower bound as detailed on p.336 of Casella and Berger's Statistical Inference:
$\frac{d}{d\theta}E[W(\bf{X})|\theta] = \int_{\chi}W(\bf{x})\left[\...
- 154
3
votes
2
answers
202
views
Variance of Rao Blackwellization for MC Estimate of Expectation
from https://arxiv.org/abs/1401.0118
If we have a function $J(X,Y)$ of two random variables $X$ and $Y$ and we want to compute the expectation $\mathbb E_{p(X,Y)}[J(X,Y)]$.
We define $\hat J(X)= \...
- 187
0
votes
0
answers
229
views
Rao-Blackwell for Poisson parameter: is $\text{E} \big( \bar{X} \mid \sum_{i = 1}^n X_i \big)$ tractable?
I searched for "rao blackwell poisson", "umvu poisson", and "umvue poisson" on this website but didn't find anything that specifically answered my question. I also searched on Google for "umvu poisson"...
- 571
5
votes
2
answers
1k
views
Finding UMVUE for a function of a Bernoulli parameter
Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the UMVUE of $(1-\theta)^{1/k}$, when $k$ is a positive integer. .
I know $\sum X_{i}$ is a ...
- 204
1
vote
0
answers
47
views
What's wrong with this proof that the sample sum is sufficient for $\theta$ in $U(0,\theta)$?
So let's say $X_i ~ U(0, \theta)$, and let's consider the two-sample sample sum, $t = \bar{X_2} = (X_1 + X_2)/2$.
So we want to show that $p(x|t) = p(x,t)/p(t) = p(x)/p(t)$ is independent of $\theta$....
- 457
3
votes
1
answer
334
views
Rao-Blackwell part of the Lehmann-Scheffe theorem
I'm trying to understand the proof of this theorem.
An unbiased estimator $T$, that is a function of a complete statistic $S$, is unique, i.e. there can't be other unbiased estimators that are ...
- 3,181
6
votes
1
answer
1k
views
Relationship between completeness and sufficiency
hopefully this isn't a duplicate of another question (at least I didn't find one).
Here is a question I have about completeness and sufficiency:
Problem: Suppose $T(x)$ is complete sufficient for $\...
- 374
7
votes
2
answers
344
views
Using all Metropolis-Hastings proposals to estimate an integral
Suppose we run the Metropolis-Hastings with target distribution $\mu$ to compute the integral $\int f\:{\rm d}\mu$. Usually, we use the estimator $$A_n:=\frac1n\sum_{i=0}^{n-1}f(X_i).$$ However, ...
- 71
2
votes
1
answer
122
views
Obtaining the expected value $E[X_{(1)} \mid\overline X = c]$
Suppose we have $X_1,\dots, X_n \overset{\text{iid}}{\sim} N(\mu = 0, \sigma^2 = 1)$, for a known $n$. And we want to calculate $E[X_{(1)} \mid \overline X = c]$, where $c \in \mathbb{R}$ is known, $...
- 21
2
votes
1
answer
328
views
maximum likelihood estimate and the UMVUE
I have been working on this question and I am little confused about how to solve it.
To evaluate the prevalence of periodontal diseases in a population, suppose that $x_i$, $i=1,\ldots,n$ are the ...
1
vote
1
answer
203
views
Sufficient Statistic and Unbiased Estimate in Exponential Family
I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he deals with sufficient statistics in exponential distributions ...
- 1,542
4
votes
1
answer
231
views
Proof of Rao Blackwellization
I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he introduces the idea of minimizing the variance of an unbiased ...
- 1,542
1
vote
1
answer
470
views
Rao-Blackwell for Minimum-Variance Unbiased Estimator
Let $X$ be an observation from a distribution with probability mass
function:$f(x;\theta) =
\left(\frac{\theta}{2}\right)^{|x|}(1-\theta)^{1-|x|}I_{\{-1,0,1\}}(x),
\, \theta \in (0,1).$ Use Rao-...
- 43
0
votes
1
answer
530
views
Rao-Blackwell Problem [closed]
Let $X_1, .. X_n$ be iid Bernoulli($\theta$).
I want to estimate $\lambda = \theta(1-\theta)$ using $\delta$ (unbiased estimator for $\lambda$) where
$\begin{cases}
\delta = 1 & X_1 =1 \...
- 21
5
votes
0
answers
297
views
Minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed
What is the minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed?
median
When we wish to estimate the median, $\mu$, of a normal distributed variable then ...
- 68.3k
9
votes
1
answer
426
views
Rao-Blackwellization in variational inference
The Black box VI paper introduces Rao-Blackwellization as a method to reduce the variance of the gradient estimator using score function, in section 3.1.
However I don't quite get the basic idea ...
- 3,429
2
votes
2
answers
499
views
Variance of Rao Blackwellization for Monte Carlo Estimate of Expectation
from https://arxiv.org/pdf/1401.0118.pdf
If we have a function $J(X,Y)$ of two random variables $X$ and $Y$ and we want to compute the expectation $\mathbb E_{p(X,Y)}[J(X,Y)]$.
We define $\hat J(X)= ...
- 662
1
vote
0
answers
83
views
Understanding Rao-Blackwell [duplicate]
From Casella and Berger:
Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ ...
- 1,340
1
vote
0
answers
43
views
Sufficient statistic for the mean of a generic distribution?
Is there such thing as a "sufficient statistics for the expectation?"
From what I understand, a sufficient statistic is defined only when there is a family of distributions parametrized by $\theta$ ...
- 175
1
vote
1
answer
214
views
How to improve an estimator for a Poisson sample
Given the statistical model $(\mathbb N_0^n, P(\mathbb N_0^n),\operatorname{Poi}(\vartheta)^{\otimes n}:\vartheta >0)$, $T(X)=X_1X_2$ is an unbiased estimator of
$\vartheta^2$. I want to improve ...
user207460
2
votes
1
answer
374
views
Why can Gibbs sampling outputs be used in Rao-Blackwellization?
I'm currently learning Chib (1995)'s method to calculate the marginal likelihood of a Bayesian model using Gibbs sampling outputs. I'm stuck in the Rao-Blackwellization step.
Suppose $\mu$ and $\phi$...
- 433
0
votes
0
answers
329
views
How to improve an estimator that is already unbiased?
I am solving a two part problem where the second part of the problem is to improve on the estimator in the first using the Rao-Blackwell theorem. In this case, the sufficient statistic is just Y ...
2
votes
1
answer
89
views
How does Rao-Blackwellisation use only $X$ in $Y | X$ to produce the result?
How does Rao-Blackwellisation use only $X$ in $Y | X$ to produce the result?
Since I read that what Rao-Blackwellisation does is find first
$$\mathbb{E}[Y | X=x] := h(x)$$
and then draws an ...
- 3,449
4
votes
2
answers
502
views
Sufficient Statistic and Maximum likelihood
This is more a conceptual question, but it seems to me that a sufficient statistic for a parameter is a concepts that applies only if we want to estimate the parameter via maximum likelihood. Is this ...
- 576
2
votes
1
answer
95
views
Reference book for practice problems on Inference
I was wondering if there is any book which has loads of problems on statistical inference.
Desired topics are
Unbiasedness
Consistency
Sufficiency
Completeness
Rao Blackwell Theorem
etc.
0
votes
0
answers
63
views
Conditional Expectation of Order Statistic [duplicate]
Given a a random sample $X_1, X_2, X_3, X_4$ and family of densities $\mathcal{P} = \left\{ f_\theta: \theta \in \Theta \right\}$, where $f_\theta(x) = \frac{1}{2}\mathbb{I}_{[\theta-1, \theta + 1]}$, ...
- 217
1
vote
1
answer
3k
views
Finding sufficient statistic for Weibull density function
I am given the follow problem and am having trouble finding the sufficient statistic.
Suppose that Y$_1$, Y$_2$, ..., Y$_n$ denote a Weibull density function, given by:
f ( y | $\theta$ ) =
Let $...
- 329
3
votes
1
answer
1k
views
Why does the MSE version of Rao-Blackwell theorem require $T$ to be a sufficient statistic?
The proof for the MSE version appears not to depend on $T$ being a sufficient statistic. I provide a minimal version here:
Let $\hat\theta,T$ be observable random variables, and let $\theta \in \...
- 1,584
1
vote
2
answers
1k
views
MVUE using factorization criterion and Rao-Blackwell theorems
Suppose that $Y_1, Y_2, ..., Y_n$ is a random sample from a
distribution with density function
$$ f(y) = \begin{cases} \theta y^{\theta - 1}\ \ \ \ 0 < y < 1, \\ 0\
\ \ \ \ \ \ \ \ \ \ ...
- 111
2
votes
1
answer
318
views
Knowing both sample mean and sample range improves estimate of the the variance?
It is known that sample variance and sample mean are independent for normally distributed variables, which means knowing sample mean does not say anything about to estimate the variance of the ...
- 689
2
votes
1
answer
727
views
Conditioning a biased estimator on a sufficient statistic
I'm afraid my awareness of the Rao–Blackwell theorem has been limited to textbook accounts and exercises, and those deal only with its application to unbiased estimators. Maybe it's properly called ...
- 8,511
11
votes
2
answers
3k
views
Rao-Blackwellization of Gibbs Sampler
I am currently estimating a stochastic volatility model with Markov Chain Monte Carlo methods. Thereby, I am implementing Gibbs and Metropolis sampling methods.Assuming I take the mean of the ...
- 545
2
votes
2
answers
469
views
Rao-Blackwellizing: Is there any difference conditional on different sufficient statistics
Suppose I have two different sufficient statistics $a_1$ and $a_2$ while $a_1$ summarizes information more efficient than $a_2$.
For example, if the sample space is $\left\{y_1,y_2,y_3,y_4,y_5,y_6\...
- 305
10
votes
2
answers
762
views
Why does the Rao-Blackwell Theorem require $\Bbb E(\hat{\theta}^2) < \infty$?
The Rao-Blackwell Theorem states
Let $\hat{\theta}$ be an estimator of $\theta$ with $\Bbb E (\hat{\theta}^2) < \infty$ for all $\theta$. Suppose that $T$ is sufficient for $\theta$, and let $\...
- 3,913
2
votes
0
answers
82
views
The normal and bernoulli distributions
I'm working on this problem and am a little stumped. I was wondering if someone could give me a hint?
$x_1,...,x_n$ are iid $N(\mu,\sigma^2)$ where $\mu$ is unknown and $\sigma$ is known. $n>3$
...
- 741
2
votes
1
answer
247
views
How does one can guarantee that any unbiased estimator is MVUE due to it containing a minimal sufficient statistic?
Sufficiency is okay. But I don't really get it why the fact guarantees it has minimal variance? Can anyone explain to me somewhat intuitively?
- 23
10
votes
2
answers
1k
views
Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$
This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is :
Let a random sample of size $n$ be taken from a distribution with
the pdf $...
- 4,648
8
votes
2
answers
3k
views
What is the necessary condition for a unbiased estimator to be UMVUE?
According to the Rao-Blackwell theorem, if statistic $T$ is a sufficient and complete for $\theta$, and $E(T)=\theta$, then $T$ is a uniformly minimum-variance unbiased estimator (UMVUE).
I am ...
- 81