0
$\begingroup$

Let ${\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{k})^{\mathrm {T} }}$ be a sample from a multivariate normal distribution ${\displaystyle \mathbf {X} \ \sim \ {\mathcal {N}}({\boldsymbol {\mu }},\,{\boldsymbol {\Sigma }})}$.

Can you calculate the mean of the output distribution after transforming with the following formula (which is the cosine similarity in vector space), assuming that ${\displaystyle \mathbf {w}}=(w_{1},\ldots ,w_{k})^{\mathrm {T} }$ is a fixed vector? $$ f(\mathbf{X}) = \frac {\sum_{i}{X_{i} w_i}} {\sqrt{\sum_i{X_i}^2}\sqrt{\sum_i{w_i}^2}} $$

$\endgroup$
4
  • $\begingroup$ It helps to simplify questions like these. For instance, a suitable invertible linear transformation will map $w$ to $(1,0,\ldots, 0)$ while changing the parameters of $X$ in a definite way, giving another multivariate Normal distribution. This simplifies the function to $f(X)=X_1/\sqrt{X_1^2+\cdots+X_k^2}.$ Now you can apply another invertible linear transformation to $X,$ reducing the function to $f(X)=X_1/\sqrt{X_1^2+X_2^2}.$ $\endgroup$
    – whuber
    Feb 16, 2023 at 15:47
  • $\begingroup$ Is calculating $E[f(\mathbf{X})]$ your question? $\endgroup$
    – Zhanxiong
    Feb 17, 2023 at 0:10
  • $\begingroup$ @whuber Thank you for pointing this out! I suppose my follow up question is how to calculate the mean of the resulting function then? $\endgroup$
    – lupus83
    Feb 17, 2023 at 9:18
  • 1
    $\begingroup$ @Zhanxiong yes, by calculation of the mean I meant the expectation 𝐸[𝑓(𝐗)] $\endgroup$
    – lupus83
    Feb 17, 2023 at 9:19

1 Answer 1

0
$\begingroup$

You may find the answer quite intuitive. Consider the sketch below. As cosine similarity deals with the angle between $\bf{X}$ and $\bf{w}$, the question boils down to projecting $\bf{X}$ on a sphere passing through the center of the distribution of $\bf{X}$, that is, through $E[\bf{X}]$. From the symmetry of the distribution of $\bf{X}$, the mean of the projected $\bf{X}$ is on the chord between $E[\bf{X}]$ and the origin. Therefore, your answer is the cosine similarity between $\bf{w}$ and $E[\bf{X}]$.

enter image description here

$\endgroup$
8
  • $\begingroup$ Could you provide a little more detail about "the symmetry of the distribution"? I don't believe this argument works, because that distribution (along with the fixed vector $w$) does not necessarily share the same similarities as the unit sphere. $\endgroup$
    – whuber
    Mar 7, 2023 at 14:31
  • $\begingroup$ I'd appreciate if you could please explain what similarities is in the context of the symmetry of a multivariate normal distribution about its mean. $\endgroup$
    – TomTim
    Mar 8, 2023 at 15:28
  • $\begingroup$ The information in the problem consists of the matrix $\Sigma,$ which describes a collection of concentric ellipsoids oriented in its (arbitrary) eigendirections; the (arbitrary) mean $\mu,$ and the (arbitrary) vector $w.$ Anything deserving of the term "symmetry" would be a transformation of these data that preserve some property of interest, which here is the cosine similarity. I am simply asking you to describe the symmetry to which you appeal in this post. I cannot find any in general. $\endgroup$
    – whuber
    Mar 8, 2023 at 15:42
  • $\begingroup$ Not sure that i understand your comment, but "a multivariate normal distribution" is spherical symmetric about its mean point. If you want to ask about the symmetry on the sphere, you may, for example, consider an infinitesimal thick sphere on which p(X) is constant, and consider the intersection between this sphere and the sphere passing through the center of the distribution of X. Please also consider this: link $\endgroup$
    – TomTim
    Mar 9, 2023 at 3:57
  • $\begingroup$ Your point of departure is incorrect: $\Sigma$ is not necessarily spherical. The only symmetries a general multivariate Normal distribution enjoys are those generated by the reflections through the mean $\mu$ determined by the (generically) distinct eigendirections. $\endgroup$
    – whuber
    Mar 9, 2023 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.