4
$\begingroup$

I am very familiar with the formula $$f_{Y}(y) = f_{X}(x(y))\left|\dfrac{\text{d}}{\text{d}y}[x(y)]\right|$$ where $X$ and $Y = g(X)$ is a one-to-one transformation of $X$. (I use $x(y)$ to mean "$x$ in terms of $y$.")

Without any justification, the book I have has provided the analogue for random vectors. It states:

If $\mathbf{Z}$ has density $f_{\mathbf{Z}}$ and $\mathbf{Y} = G(\mathbf{Z})$, then the density of $\mathbf{Y}$ is $$f_{\mathbf{Y}}(\mathbf{y}) = f_{\mathbf{Z}}\left(G^{-1}(\mathbf{y})\right)\left|\det\left(\text{d}G^{-1}\right)\right|$$ where $\text{d}G^{-1}$ is the derivative (matrix of partial derivatives) of $G^{-1}$ evaluated at $\mathbf{y}$.

None of the definitions have been provided in this textbook to prove this formula. My initial instinct is that I need to find the CDF of $\mathbf{Y}$ and take derivatives, but how would one define the CDF of a random vector and, how does one take derivatives with respect to a vector so that it is relevant to the problem?

Also, my guess is that the matrix of partial derivatives refers to what is found in the link here. I'm also surprised that conditions on $G$ were not given; I assume $G$ has to be something like one-to-one except in the random vector analogue?

In short: I have no idea how to justify the formula for random vectors since none of the machinery used to prove it has been developed, and it is expected that I use the result without proof. Could someone guide me through a proof of the formula?

Edit: It looks like Casella and Berger offer a brief explanation of this equation. Let $\mathbf{X} = (X_1, \dots, X_n)$ be a random vector with pdf $f_{\mathbf{X}}$, and let $\mathcal{A} = \{\mathbf{x}:f_{\mathbf{X}}(\mathbf{x}) > 0\}$. Let $\mathbf{U} = (U_1, \dots, U_n)$ be a random vector such that $U_k = g_k(X_1, \dots, X_n)$ for all $k$. This transformation is one-to-one from $A_i$ onto $\mathcal{B}$ (I believe this is the Borel $\sigma$-algebra) for each $i$, and furthermore, their inverses can be found, such that (letting $(u_1, \dots, u_n) = \mathbf{u}$) $x_k = h_{ki}(\mathbf{u})$ and $u_k = g_k(\mathbf{x})$. Then the PDF of $\mathbf{U}$ is $$f_{\mathbf{U}}(\mathbf{u}) = \sum\limits_{i}f_{\mathbf{X}}(h_{1i}(\mathbf{u}), \dots, h_{ni}(\mathbf{u}))|J_i|$$ where $$J_i = \left|\dfrac{\partial x_j}{\partial u_i}\right|\text{.}$$ I've probably translated this wrong. Casella and Berger say to look at Searle 1982, which is Matrix Algebra Useful for Statistics, which I imagine just provides the linear algebra background. No proof is given.

Edit2: It looks like Bickel and Doksum's Mathematical Statistics: Basic Ideas and Selected Topics has a proof.

$\endgroup$
  • 1
    $\begingroup$ Look for Jacobians in your textbook for justifications. Basically, for a small neighborhood $S_Y$ containing $\mathbf y$, $P\{\mathbf Y \in S_Y\} \approx f_{\mathbf Y}(\mathbf y)|S_Y|$ where $|S_Y|$ is the volume of the neighborhood and this must be the same as $P\{\mathbf Z \in S_Z\} = f_{\mathbf Z}((G^{-1}(\mathbf y)|S_Z|$ where $S_Z$ is the corresponding neighbor hood for $\mathbf Z$ and its volume is $|S_Z|$. But the ratio of the volumes is given by the Jacobian (consult almost any multivariate calculus book). $\endgroup$ – Dilip Sarwate Jun 5 '15 at 4:09
  • 3
    $\begingroup$ I have posted a full explanation of the underlying theory at stats.stackexchange.com/a/154298 specifically for the purpose of answering questions like this one. If you believe there is anything missing that you would like to see, then please let me know. $\endgroup$ – whuber Jun 5 '15 at 15:56
2
+50
$\begingroup$

whuber gave an incredibly detailed description of the mechanics that go into this kind of calculation, so all I can add is some intuition for why a formula like that shows up.

Suppose you have a linear mapping $L: R^n \rightarrow R^n$. This maps the unit cube in the domain to a parallelepiped in the range whose corners are given by all unit linear combinations of

$$ 0, L(1, 0, \ldots, 0), L(0, 1, \ldots, 0), \ldots, L(0, 0, \ldots, 1) $$

The geometric interpretation of the $\det(L)$ is that it is the volume of this image parallelepiped. I believe you can actually prove this using whuber's formalism if you're careful enough.

Now, suppose you have a general function $f: R^n \rightarrow R^n$; let's say it's one-to-one to avoid edge cases. Suppose we have some shape $\Omega$ in the domain, and we want to calculate the volume of its image $f(\Omega)$. Let's cut $\Omega$ up into tiny, non-overlapping cubes, say

$$ \Omega = C_1 \cup C_2 \cup \cdots \cup C_N $$

Then, since $f$ is one-to-one, the image cubes don't overlap either, and

$$ f(\Omega) = f(C_1) \cup f(C_2) \cup \cdots \cup f(C_n) $$

So we really need to know the volume of the image of a tiny cube under $f$, and then we can just sum the volumes up over the dissection of $\Omega$ into cubes.

In comes the derivative. For a function $f: R^n \rightarrow R^n$ and a point $p$ in its domain, the derivative of $f$ at $p$, which we notate by $d_p f$, is a linear mapping. This linear mapping is the directional derivative mapping: for a given vector $v$ based at $p$, it returns a vector based at $f(p)$ that is the directional derivative of $f$ in the direction of $v$. There is a general fact that very close to $p$ the function $f$ is well approximated by a linear function (this is what it means to be differentiable). What is this linear function that best approximates $f$? It's $d_p f$ of course!

So now suppose that our tiny cube $C_i$ has a corner at $p$. If the cube is small enough, the non-linear effects introduced into $f(C_i)$ by $f$ will be very small, so $f(C_i)$ will look quite like a small cube as well. That is $f(C_i)$ will look like the image of $C_i$ under the derivative $d_p f$ of $f$ at $p$. But we know how volumes transform under linear maps, they transform by the determinant! So:

$$vol (f(C_i)) \approx det(d_p f) vol(C_i) $$

Summing these up over all the tiny cubes, we get:

$$ \int_{f(\Omega)} dq = \int_{\Omega} det(d_p f) dp $$

Which is the usual multi-variable integral formula, only viewed in a mirror.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.