# Entropy: Given y=Ax, then H[y] = H[X] + ln |A|

The entropy of H[y] is given by:

$$H[y] = - \int p_y(\mathbf{y})~ \ln p_y(\mathbf{y})~ d\mathbf{y}$$

Now, suppose that I want to make a linear transformation of vector $$\mathbf{y}$$ to change the variables of the integration to vector $$\mathbf{x}$$ according to:

$$\mathbf{y}=\mathbf{A}\mathbf{x}$$

Further, suppose that vector $$\mathbf{x}$$ is a continuous random variable with distribution $$p_x(x)$$ and corresponding entropy H[x].

What is H[y] in terms of H[x]?

Step 1:

$$H[\mathbf{y}] = - \int p_y(\mathbf{y})~ \ln p_y(\mathbf{y})~ d\mathbf{y}$$

Step 2: $$H[\mathbf{y}] = - \int \bigg(\frac{p_x(\mathbf{x})}{|\mathbf{A}|}\bigg) \ln \bigg( \frac{p_x(\mathbf{x})}{|\mathbf{A}|}\bigg)~ \bigg| \frac{\partial y}{\partial x} \bigg| d\mathbf{x}$$

Step 3: $$H[\mathbf{y}] = -\int p_x(\mathbf{x}) \ln \bigg(\frac{p_x(\mathbf{x})}{|\mathbf{A}|}\bigg)d\mathbf{x}$$

Step 4: $$H[\mathbf{y}] = -\int p_x(\mathbf{x}) \ln p_x(\mathbf{x}) d\mathbf{x} - \int p_x(\mathbf{x}) \ln\bigg( \frac{1}{|\mathbf{A}|}\bigg)d\mathbf{x}$$

Step 5: $$H[\mathbf{y}] = H[\mathbf{x}] + ln |\mathbf{A}|$$

My question is as follows:

What happened between step 1 and step 2? I wanted to see the details of this step broken down so that i could follow it. The other steps I can follow without any problems...

Under a nonlinear change of variable, a probability density transforms differently from a simple function, due to the Jacobian factor.

For instance, if we consider a change of variables $$x=g(y)$$, then a function $$f(x)$$ becomes $$\tilde{f}(x) = f(g(x))$$.

Now consider a probability density $$p_x(x)$$ that corresponds to a density $$p_y(y)$$ with respect to a new variable y, where the suffixes denote the fact that $$p_x(x)$$ and $$p_y(y)$$ are different densities.

Observations falling in the range $$(x, x + \delta x)$$ will, for small values of $$\delta x$$, be transformed into the range $$(y, y + \delta y)$$ where:

$$p_x(x) \delta x = p_y(y) \delta y$$

and hence:

$$p_y(y) = p_x(x) \bigg| \frac{dx}{dy}\bigg|$$