2
$\begingroup$

Question:

Given, $X\sim N(0,\sigma^2)$. By means of conditional approach show that $|X|$ is a sufficient estimator for $\sigma^2$.

My Attempt:

This problem is very easy if we use Fisher–Neyman Factorisation theorem. But in this problem we have to use the basic definition of Sufficient Statistics, i.e,

A statistic $t = T(X)$ is sufficient for underlying parameter $\theta$ precisely if the conditional probability distribution of the data $X$, given the statistic $t = T(X)$, does not depend on the parameter $θ$.

$X\sim N(0,\sigma^2)$, here parameter, $\theta=\sigma^2$. So we have to show that $P_{\sigma^2}(X\le\ x\mid T(X)\le\ t)$ is independent of $\sigma^2$.

For $x\in\ (-t,t]$,

\begin{align} P_{\sigma^2}(X\le\ x \mid T(X)\le\ t)&= \frac{P_{\sigma^2}(X\le\ x,T(X)\le\ t)}{P_{\sigma^2}(T(X)\le\ t)} \\&=\frac{P_{\sigma^2}(X\le\ x,|X|\le\ t)}{P_{\sigma^2}(|X|\le\ t)} \\&=\frac{P_{\sigma^2}(-t\le\ X\le\ x)}{P_{\sigma^2}(-t\le\ X\le\ t)} \end{align}

But how to show that this expression is free of $\sigma^2$?

$\endgroup$
2
  • $\begingroup$ I like this approach because you might find it surprising: Let $U$ be a Rademacher variable independent of $X.$ Because $\Pr(U=1)=\Pr(U=-1)=1/2,$ the variable $Y = |X|U$ has a Normal$(0,\sigma^2)$ distribution. You know that $Y$ is a sufficient estimator for $\sigma^2,$ whence (because $Y$ is derived from $|X|$) so is $|X|.$ $\endgroup$
    – whuber
    Commented Mar 11, 2023 at 18:21
  • $\begingroup$ If we can use the fact that $\frac{X}{|X|}$ and $|X|$ are independently distributed, then $$P\left(X\le x\mid |X|=t\right)=P\left(\frac{X}{|X|}\le \frac{x}{t}\mid |X|=t\right)=P\left(\frac{X}{|X|}\le \frac{x}{t}\right).$$ And this quantity is free of $\sigma^2$ simply because the distribution of $\frac{X}{|X|}=\frac{X/\sigma}{|X/\sigma|}$ is free of $\sigma^2$. $\endgroup$ Commented Jun 9 at 18:55

2 Answers 2

2
$\begingroup$

The problem of your derivation is that you misunderstood the concept of conditional distribution. It is not $P_\sigma(X \leq x |T(X) \leq t)$ -- it should be $P_\sigma(X \leq x | T(X) = t)$. For a thorough discussion of the latter notation, see this answer.

To derive the correct conditional distribution, intuitively, given $|X| = t > 0$, then $X$ can only take value $t$ or $-t$. Therefore, for any $x \in \mathbb{R}$, the event $[X \leq x] \cap [|X| = t]$ is: \begin{align} \begin{cases} \varnothing & x < -t, \\ [X = -t] & -t \leq x < t, \\ [|X| = t] & x \geq t. \end{cases} \end{align} By symmetry of $N(0, \sigma^2)$, this implies the conditional distribution of $X$ given $|X| = t$ is \begin{align} P[X \leq x | |X| = t] = \begin{cases} \frac{1}{2}I_{[-x, \infty)}(t) & x \leq 0, \\ I_{(0, x]}(t) + \frac{1}{2}I_{(x, \infty)}(t) & x > 0. \end{cases} \tag{1} \end{align} Therefore the conditional distribution of $X$ given $|X|$ does not depend on $\sigma^2$, hence $|X|$ is sufficient for the distribution family $\{N(0, \sigma^2): \sigma > 0\}$.


To prove $(1)$ rigorously, first rewrite $(1)$ as \begin{align} P[X \leq x | |X|] = \begin{cases} \frac{1}{2}I_{[|X| \geq -x]}(\omega) & x \leq 0, \\ I_{[|X| \leq x]}(\omega) + \frac{1}{2}I_{[|X| > x]}(\omega) & x > 0. \end{cases} \tag{2} \end{align} Since the right-hand side of $(2)$ is obviously $\sigma(|X|)$-measurable, it suffices to show for any generic $\sigma(|X|)$-set $[|X| \leq t]$, where $t > 0$, it holds that (these are two defining relations of the measure-theoretic conditional probability. For more details, refer to, for example, Equation (33.8) in Probability and Measure by Patrick Billingsley): \begin{align} P[[X \leq x]\cap [|X| \leq t]] = \int_{[|X| \leq t]}P[X \leq x ||X|]dP. \tag{3} \end{align}

When $x \leq 0$, the left-hand side of $(3)$ is $(\Phi_\sigma(x) - \Phi_\sigma(t))I_{[-t, 0]}(x)$, while the right-hand side of $(3)$ is \begin{align} \frac{1}{2}P[|X| \leq t, |X| \geq -x] = (\Phi_\sigma(x) - \Phi_\sigma(t))I_{[-t, 0]}(x). \end{align} Hence $(3)$ holds.

When $x > 0$, the left-hand side of $(3)$ is \begin{align} (\Phi_\sigma(x) - \Phi_\sigma(-t))I_{(0, t)}(x) + (\Phi_\sigma(t) - \Phi_\sigma(-t))I_{[t, \infty)}(x), \end{align} while the right-hand side of $(3)$ is \begin{align} & P[|X| \leq t, |X| \leq x] + \frac{1}{2}P[|X| \leq t, |X| > x] \\ =& \left[P[|X| \leq x] + \frac{1}{2}P[x < |X| \leq t]\right]I_{(0, t)}(x) + P[|X| \leq t]I_{[t, \infty)}(x) \\ =& (\Phi_\sigma(t) - \Phi_\sigma(-x))I_{(0, t)}(x) + (\Phi_\sigma(t) - \Phi_\sigma(-t))I_{[t, \infty)}(x)\\ =& (\Phi_\sigma(x) - \Phi_\sigma(-t))I_{(0, t)}(x) + (\Phi_\sigma(t) - \Phi_\sigma(-t))I_{[t, \infty)}(x). \end{align} Hence $(3)$ holds. This completes the proof.

$\endgroup$
1
  • $\begingroup$ This is absolutely precise. First showing with the symmetry of the gaussian distribution and finally deducing it in a rigorous manner which I like especially that it can be a good example of how conditional probability works. $\endgroup$ Commented Mar 11, 2023 at 19:10
1
$\begingroup$

$[\rm I]$ notes for $T(\mathbf X) $ to be sufficient for $\theta, $

$$ \mathbb P_\theta(\mathbf X=\mathbf x\mid T(\mathbf X) =T(\mathbf x))=\frac{p(\mathbf x\mid\theta)}{q(T(\mathbf x)\mid \theta)} \tag 1\label 1$$ should be constant as a function of $\theta.$

It is well-known that if $X\sim\mathrm N(\mu, \sigma^2), $ then $|X|\sim\mathrm{FoldedN}(\mu, \sigma^2). $

Here $X\sim\mathrm N(0, \sigma^2), ~n=1.$ So, evaluating $\eqref 1$

\begin{align}\frac{p(x\mid\sigma^2)}{q(|x| \mid \sigma^2)}&=\frac{(2\pi\sigma^2)^{-1/2}\exp\left[-\frac{x^2}{2\sigma^2}\right]}{(2\pi\sigma^2)^{-1/2}\left(\exp\left[-\frac{x^2}{2\sigma^2}\right]+\exp\left[-\frac{x^2}{2\sigma^2}\right]\right)}\\&=\frac12\end{align}

shows $|X|$ is sufficient for $\sigma^2.$


Reference:

$\rm [I]$ Statistical Inference, George Casella, Roger L. Berger, Wadsworth, $2002, $ sec. $6.2.1, $ p. $273.$

$\endgroup$
4
  • 1
    $\begingroup$ straight to the point, +1! $\endgroup$
    – utobi
    Commented Mar 11, 2023 at 19:22
  • $\begingroup$ While I understand this is the standard way of approaching the sufficiency problem without resorting to measure theory, the validity $(1)$ needs to be interpreted carefully when $X$ is continuous. Rigorously speaking, the LHS of $(1)$ is always $0$ when $X$ is continuous hence cannot equal to the RHS of it(I checked [1], where the author stated We will do our calculations in the discrete case and will point out analogous results that are true in the continuous case). $\endgroup$
    – Zhanxiong
    Commented Mar 11, 2023 at 20:15
  • $\begingroup$ @Zhanxiong, yes. You are right. Casella's book was for undergrad to early grad studies and is infamous for lacking measure-theoretic treatments. What should be valid in this case that as the authors indicated the ratio has to be constant as function of the concerned parameter. $\endgroup$ Commented Mar 11, 2023 at 20:20
  • $\begingroup$ Theorem 6.2.2 in [1] concerns the ratio (RHS of (1)) directly without mentioning the LHS of (1) in the continuous case. However, if we apply this theorem, then it is more or less the same as the factorization theorem, in my opinion. $\endgroup$
    – Zhanxiong
    Commented Mar 11, 2023 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.