1
$\begingroup$

It is a general rule that for multivariate data $\boldsymbol{X}$ and a matrix $\boldsymbol{A}$, their entropy is

$$h(\boldsymbol{A} \boldsymbol{X}) = h(\boldsymbol{X}) + \ln |\det \boldsymbol{A}|$$ (see https://en.wikipedia.org/wiki/Differential_entropy#Properties_of_differential_entropy).

What is the rule for the entropy of $\boldsymbol{X}$ multiplied by a vector $\boldsymbol{b}$?

$$ h(\boldsymbol{X} \boldsymbol{b} ) = ?$$

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Emulate the analysis given at stats.stackexchange.com/questions/415435. It's unclear what you mean by your "also" question, since all of this is explicitly about distributions. $\endgroup$
    – whuber ♦
    Sep 25, 2020 at 22:34
  • $\begingroup$ thanks, edited the last part. the question here is what the non-reduced entropy solution is of multiplying a multivariate dataset by a vector of weights, each corresponding to a univariate component of the multivariate. Not about the effect of shift or scale. Your link is all about univariate entropy only $\endgroup$
    – develarist
    Sep 25, 2020 at 22:55
  • $\begingroup$ The rule you linked assumes X is a random vector, not a random matrix. So, by Xb, do you mean the inner product of X and b? $\endgroup$
    – PedroSebe
    Sep 26, 2020 at 1:15
  • 1
    $\begingroup$ My link is about how to work with entropy. The method there extends, with no essential change, to multivariate problems. $\endgroup$
    – whuber ♦
    Sep 27, 2020 at 12:35

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.