2
$\begingroup$

My question concerns Exercise 9.10 of Statistical Inference by Casella and Berger: On page 428 the authors say

In general, suppose the pdf of a statistic $T$, $f(t|\theta)$, can be expressed in the form $$ f(t|\theta) = g(Q(t,\theta)) \left| \frac{\partial}{\partial t} Q(t,\theta)\right| $$

for some function $g$ and some monotone function $Q$ (monotone in $t$ for each $\theta$). Then Theorem 2.1.5 can be used to show that $Q(T,\theta)$ is a pivot (see Exercise 9.10).

Exercise 9.10 just asks for a proof of this statement.

For reference, here is Theorem 2.1.5 (slightly paraphrased):

Theorem 2.1.5 Let $X$ have pdf $f_X(x)$ and let $Y = g(X)$, where $g$ is a monotone function. Let $\mathcal{X}$ be the support of $f_X$ and let $\mathcal{Y} = g(\mathcal{X})$. Suppose that $f_X(x)$ is continuous on $\mathcal{X}$ and that $g^{-1}(y)$ has a continuous derivative on $\mathcal{Y}$. Then the pdf of $Y$ is given by $$ f_Y(y) = f_X(g^{-1}(y)) \left| \frac{d}{dy}g^{-1}(y) \right|, \quad y \in \mathcal{Y}. $$

I got a bit stuck on this problem so I perused online and came across the following solution: enter image description here

I'm a bit confused here. Specifically, how was the following line obtained? $$ f_Y(y) = g(y) \left| \frac{d Q^{-1}(y;\theta)}{dt}\right|^{-1} \left| \frac{d Q^{-1}(y;\theta)}{dt}\right| $$ obtained?

Edit 5/7/22: Thanks to @whuber for the excellent proof. However, I'm still confused about why the following argument leads to a seeming contradiction. To hopefully make things clearer, I introduce the following notation:

  • Let $\Theta$ be the parameter space.

  • Let $\mathbb{S}^n (\subseteq \mathbb{R}^n)$ denote the sample space for a random sample of size $n$.

  • Let $s: \mathbb{S}^n \rightarrow \mathcal{T}$ where $\mathcal{T} := s(\mathbb{S}^n) \subseteq \mathbb{R}^n$.

  • Let $T = s(\mathbf{X})$ and let $f_T(t|\theta)$ be the pdf of $T$.

  • Let $q: \mathcal{T} \times \Theta \rightarrow \mathbb{R}$ be a function which is monotone in $t$ for each $\theta \in \Theta$.

  • For each $\theta \in \Theta$, let $q_{\theta}: \mathcal{T} \rightarrow \mathbb{R}$ be given by $q_{\theta}(t) := q(t,\theta)$ for all $t \in \mathcal{T}$.

  • Let $g: q(\mathcal{T} \times \Theta) \to \mathbb{R}$ be a fixed function.

  • For each $\theta \in \Theta$, let $Y_{\theta} = q_{\theta}(T)$ and let $f_{Y}(y|\theta)$ be the pdf of $Y_{\theta}$.

We are given that $$ f_{T}(t|\theta) = g(q(t,\theta)) \left| \frac{\partial}{\partial t} q(t,\theta) \right| \qquad \forall (t,\theta) \in \mathcal{T} \times \Theta \hspace{3cm} $$

or equivalently, $$ f_T(t|\theta) = g(q_{\theta}(t)) \left| \frac{d}{dt}q_{\theta}(t) \right| \qquad \forall (t,\theta) \in \mathcal{T} \times \Theta. \hspace{3cm} (1) $$

Now since $Y_{\theta} = q_{\theta}(T)$ where $q_{\theta}$ is monotone (by assumption), $q_{\theta}$ has a well-defined inverse $q_{\theta}^{-1}$. Then applying Theorem 2.1.5 and (1), we obtain

\begin{align*} f_{Y}(y|\theta) &= f_T(q_{\theta}^{-1}(y)) \left| \frac{d}{dy} q_{\theta}^{-1}(y)\right| && (\text{Change of Vars Theorem}) \\[5pt] &= \left(g(q_{\theta}(q_{\theta}^{-1}(y))) \left|\frac{d}{dy} q_{\theta}(q_{\theta}^{-1}(y)) \right| \right) \left| \frac{d}{dy} q_{\theta}^{-1}(y)\right| && (\text{Applying (1)}) \\[5pt] &= \left( g(y) \left|\frac{d}{dy}(y) \right| \right) \left| \frac{d}{dy} q_{\theta}^{-1}(y)\right| && (\text{Simplifying}) \\[5pt] &= g(y) \left| \frac{d}{dy} q_{\theta}^{-1}(y)\right|. && (\text{Simplifying}) \end{align*}

But @whuber's answer shows that $f_Y \equiv g$, so we would need $\left|\frac{d}{dy} q_{\theta}^{-1}(y) \right| \equiv 1$. But this appears to be a contradiction because there was no such restriction on $q_{\theta}$ to begin with...where in the above argument did I go wrong?

Any insights, or alternative solutions, would be greatly appreciated.

$\endgroup$
9
  • 1
    $\begingroup$ Which is the "first equality"? $\endgroup$
    – whuber
    May 2 at 20:00
  • 1
    $\begingroup$ That's a standard formula for changing the variable of a density (making allowances for a truly abominable notation--it's discouraging to see a text try to differentiate a function of $y$ and $\theta$ with respect to $t$!). See stats.stackexchange.com/questions/4220 for why it's needed (and what it means) and see any post that mentions probability element for examples. $\endgroup$
    – whuber
    May 2 at 20:26
  • 1
    $\begingroup$ You might do better by changing the notation to something that is less ambiguous and more rigorous and then re-doing the calculation. I think the notation is really interfering with both the algebra and the understanding. $\endgroup$
    – whuber
    May 2 at 21:03
  • 1
    $\begingroup$ It's all ambiguous and inconsistent. For instance, sometimes you drop the "$|\theta$" from $f;$ it's unclear what expressions are functions and which are being kept constant; there's no distinction made in your notation between partial derivatives and total derivatives; and it's even unclear what substitutions or definitions are being made--you seem to define $y$ in terms of a function of itself, for instance. All this can easily lead to hard-to-detect mistakes. $\endgroup$
    – whuber
    May 2 at 21:43
  • 1
    $\begingroup$ Atrocious, indeed. $\endgroup$
    – Xi'an
    May 3 at 6:14

2 Answers 2

3
$\begingroup$

In an effort to clarify the notation I have arrived at an equivalent demonstration based on the distribution functions rather than the densities. Let's see how this plays out.


What I aim to achieve is a notation that is at once rigorous and clear. The methods I use include

  1. Make consistent typographic distinctions among random variables, distribution functions, densities, and parameters. I have chosen a conventional system:

    • $\mathbf T$ is a random variable.
    • $F$ is a distribution function.
    • $f$ is a generic function, including a density function.
    • $\theta$ is a parameter.
    • $t$ is a real variable.

    This convention offers no typographic distinction between functions and variables, but the "$f(t)$" notation makes their roles clear.

  2. Distinguish functions, such as $f,$ from their values at arguments like $f(t).$ Thus, when $f$ is a real-valued function, $f(t)$ is not a function: it's a real number.

  3. Make the differential elements, like $\mathrm{d}t,$ explicitly appear in expressions involving densities, because these are essential to transforming them correctly.

  4. Name parameters like $\theta$ explicitly where it will help, but make it clear they are not going to be differentiated or integrated: they are just along to designate members of a family of distributions. To this end I subscript them to get them out of the way.


Let $f_{\mathbf T;\theta}$ be the pdf of the statistic $\mathbf T$ when the parameter is $\theta.$ This means $f_{\mathbf T;\theta}$ is the derivative of the distribution function of $\mathbf T,$ or equivalently

$$\mathrm{d} F_{\mathbf T;\theta}(t) = f_{\mathbf T;\theta}(t)\,\mathrm{d}t.$$

Suppose that for each parameter value $\theta$ the probability element on the right hand side can be expressed in the form

$$f_{\mathbf T;\theta}(t)\,\mathrm{d}t = g(q_\theta(t))\, \mathrm{d}q_\theta(t) = \mathrm{d}\left(G\circ q_\theta\right)(t)\tag{*}$$

for a family of functions $q_\theta$ and a fixed function $G$ with derivative $G^\prime = g.$ (Notice that changing $G$ by subtracting any constant value will not alter this relationship with $g.$ We will exploit this at the end.)

The preceding sets of equations give

$$\mathrm{d}F_{\mathbf T;\theta} = \mathrm{d}\left(G\circ q_\theta\right)(t).$$

I wish to emphasize the clarity and simplicity of this equation, because it is the crux of the matter: it says the family of densities $\mathrm d F_\theta$ is obtained by transforming a fixed distribution $G$ by a family of functions $\{q_\theta\}.$ This insight, which is suggested by the new notation, was buried in the distracting, rococo details of the original statement of the theorem.

Integrating gives

$$F_{\mathbf T;\theta}(t) = \int_{-\infty}^t \mathrm{d}F_{\mathbf T;\theta}(x)\,\mathrm{d}x = \int_{-\infty}^t \mathrm{d}\left(G\circ q_\theta\right)(x) = G(q_\theta(t)) - \lim_{s\to-\infty} G(q_\theta(s)).$$

Write $C_\theta$ for that (annoying) limit on the right hand side. Later it will go away, but only after we make some additional assumptions.

We have enough information to compute the distribution of $\mathbf Y = q_\theta(\mathbf T)$ provided each $q_\theta$ is monotonically increasing. (It would be overkill, although strictly speaking required, to write $\mathbf Y = \mathbf Y_\theta.$ For that matter we should be writing $\mathbf T = \mathbf T_\theta,$ but I hope you agree all this subscripting would be redundant: it's enough to subscript their distribution and density functions with $\theta.$)

This assumption on the $q_\theta$ implies there exists a corresponding family of monotonic right inverse functions $q_\theta^{-1}$ for which

$$q_\theta(q^{-1}_\theta(y)) = y$$ for all $y.$

Invoke the definition of the distribution function twice to deduce

$$\begin{aligned} F_{\mathbf Y;\theta}(y) & := \Pr(q_\theta(\mathbf T)\le y) \\ &= \Pr(\mathbf T \le q_\theta^{-1}(y))\\ & =: F_{\mathbf T;\theta}(q_\theta^{-1}(y)) \\ &= G(q_\theta(q_\theta^{-1}(y)) - C_\theta \\ &= G(y) - C_\theta. \end{aligned}$$

This implies $C_\theta$ does not depend on $\theta,$ because (for instance)

$$0 = \lim_{y\to -\infty} F_{Y;\theta}(y) = \lim_{y\to-\infty} G(y) - C_\theta$$

shows $C_\theta = \lim_{y\to-\infty} G(y) =: C,$ say. We may therefore replace $G$ by the function $G-C$ without changing any of the foregoing calculations and conclude that

The distribution function of $q_\theta(\mathbf T)$ is $G$ no matter what value the parameter $\theta$ might have.

That's what it means to be a "pivot."

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed proof and explanation! I will take some time to digest this. Also, I've edited my post with a new proof which hopefully makes things more rigorous and less ambiguous. $\endgroup$
    – Leonidas
    May 3 at 15:32
  • $\begingroup$ I've taken a closer look at your proof--it's very nice and makes sense! Thanks for taking the time to write up the details. I have one remaining question: where did I go wrong in my proof attempt? (I just edited my proof because I realized the previous version was nonsense. It should hopefully be less non-nonsensical now.) It seems to be flawed, and I would very much like to know where flaw lies. $\endgroup$
    – Leonidas
    May 7 at 15:19
1
$\begingroup$

In the first equation, the solution writer applied the change of variable formula to change from a density in $t$ to one in $Y = Q(t; \theta)$, as well as applying the given information about the format of $f_Y$. The complete statement of the change of variable formula can be found in Theorem 2.1.5 of Casella & Berger.

$\endgroup$
1
  • $\begingroup$ Thanks for your answer. I am familiar with the change of variables formula. I'm still not seeing how the formula leads to $f_Y(y) = g(y)$. Could you spell out the details? $\endgroup$
    – Leonidas
    May 2 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.