1
$\begingroup$

It is a general rule that for multivariate data $\boldsymbol{X}$ and a matrix $\boldsymbol{A}$, their entropy is

$$h(\boldsymbol{A} \boldsymbol{X}) = h(\boldsymbol{X}) + \ln |\det \boldsymbol{A}|$$ (see https://en.wikipedia.org/wiki/Differential_entropy#Properties_of_differential_entropy).

What is the rule for the entropy of $\boldsymbol{X}$ multiplied by a vector $\boldsymbol{b}$?

$$ h(\boldsymbol{X} \boldsymbol{b} ) = ?$$

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Emulate the analysis given at stats.stackexchange.com/questions/415435. It's unclear what you mean by your "also" question, since all of this is explicitly about distributions. $\endgroup$ – whuber♦ Sep 25 '20 at 22:34
  • $\begingroup$ thanks, edited the last part. the question here is what the non-reduced entropy solution is of multiplying a multivariate dataset by a vector of weights, each corresponding to a univariate component of the multivariate. Not about the effect of shift or scale. Your link is all about univariate entropy only $\endgroup$ – develarist Sep 25 '20 at 22:55
  • $\begingroup$ The rule you linked assumes X is a random vector, not a random matrix. So, by Xb, do you mean the inner product of X and b? $\endgroup$ – PedroSebe Sep 26 '20 at 1:15
  • 1
    $\begingroup$ My link is about how to work with entropy. The method there extends, with no essential change, to multivariate problems. $\endgroup$ – whuber♦ Sep 27 '20 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.