0
$\begingroup$

Preliminaries

Suppose we have a random variable $X$ with density $f$ and a suitably smooth function $g: \mathbb{R} \mapsto \mathbb{R}$. The random variable $Y = g(X)$ also has a density function $h$.

Examples

I've been imagining the graphs of functions, and the resulting densities $h$, when $f$ is uniform for various choices of $g$. I only use the uniform distribution because I find it easy to interpret and illustrate the basic pattern, but in general I have noticed that both the input density $f$ and the choice of function $g$ have an influence on what $h$ will look like. I also sample over a symmetric interval about zero, as this choice also has an effect. Unlike many actual regression models, I have not included any error terms in $g$ because this doesn't seem to affect the result beyond introducing some instability into the resulting distribution shape. At the core, what I want to show below is that $\frac{dg}{dx}$ seems to be related to $h$.

Linear

For example, if $g$ is a line then $h$ will be uniform. Below is a plot of a such a sample where $g(x) = 3x+1$. Note that $\frac{dg}{dx} = 3$, a constant.

enter image description here

Quadratic

Now let us consider $Y = X^2$. Notice that $\frac{dg}{dx} = 2x$, and that the highest density appears to be around $x=0$ where this derivative is at its minimum.

enter image description here

Sigmoidal

Now consider this non-convex function, the standard sigmoid function, $Y = \frac{1}{1 + e^{-X}}$. Here the derivative $\frac{dg}{dx} = \frac{e^X}{(1 + e^{X})^2}$. The density $h$ seems to be at its largest when the derivative is at its lowest in absolute value, and vice versa.

enter image description here

Hypothesis

Some kind of antimonotonic relationship exists between $\frac{dg}{dx}$ and $h$. Maybe even something similar to $h = \left| \frac{dg}{dx} \right|^{-1}$, but that is unfettered speculation.

Question

I would be shocked if this has not been noticed by others. Rather I suspect it is simply a gap in my knowledge of mathematical statistics. So I humbly ask for some instruction on the relationship between $\frac{dg}{dx}$ with $h$.

$\endgroup$
4
  • 1
    $\begingroup$ Is this what you’re looking for? stats.stackexchange.com/q/239588/290854 $\endgroup$
    – jtb
    Commented Jul 31, 2022 at 19:04
  • $\begingroup$ @JoshBone Nice find! That definitely improves my understanding. That certainly explains the linear and sigmoid cases since they are strictly monotonic. There is still something to be explained about non-monotonic functions such as the quadratic. $\endgroup$
    – Galen
    Commented Jul 31, 2022 at 19:20
  • $\begingroup$ @JoshBone Wikipedia describes the non-monotonic case to be quite similar but involves summing this relation over all solutions of $g(x)=y$. $\endgroup$
    – Galen
    Commented Jul 31, 2022 at 20:47
  • $\begingroup$ Here is a related Python gist. $\endgroup$
    – Galen
    Commented Oct 17, 2022 at 1:42

1 Answer 1

2
$\begingroup$

Supposing that $g$ is a monotonic function, then the link posted by Josh Bone in the comments of the question applies. That is the relation

$$p_Y(y) = p_X(x) \left| \frac{dx}{dy} \right|$$

holds (under suitable smoothness). This describes the linear and sigmoidal functions in the question nicely.

In the non-monotonic case of $Y = X^2$, a similar equation

$$p_Y(y) = \sum_{k=1}^{n(y)} \left| \frac{d}{dy} g_k^{-1}(y) \right| p_X(g_k^{-1}(y))$$

holds where $n(y)$ is the number of solutions to $g(x)=y$ exists. In the case of a quadratic equation, there will be two such functions due to the fundamental theorem of algebra.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.