I was recently reading a research paper on Probabilistic Matrix Factorization and the authors were picking a random vector from a spherical gaussian distribution

ui ∼N (0,λ−1IK).

Where lambda is a regularization parameter and IK is Kth dimensional identity matrix. They provided no details on how this is actually done.

Can any one point me in the right direction for achieving that?


migrated from mathoverflow.net Nov 13 '16 at 0:18

This question came from our site for professional mathematicians.

  • $\begingroup$ Are you looking for an explanation of the notation, theory, algorithms, or software? $\endgroup$ – whuber Nov 13 '16 at 1:26

Any mean zero Gaussian random vector $\newcommand{\bR}{\mathbb{R}}$ on $ X=(X_1,\dotsc, X_n)\in\bR^n$ is uniquely determined by its covariance matrix $C$. This is a symmetric $n\times n$ matrix with entries $\newcommand{\bE}{\mathbb{E}}$

$$ C_{ij}=\bE[X_iX_j],\;\;1\leq i,j\leq n, $$

$\bE=$ expectation. The matrix $C$ is positive semidefinite, i.e., $\newcommand{\bx}{\boldsymbol{x}}$ $\newcommand{\by}{\boldsymbol{y}}$

$$ (C\bx,\bx)\geq 0,\;\;\forall \bx\in\bR^n. $$

To simulate (sample) such a random vector proceed as follows.

  1. Compute the square root of $C$.This is the unique symmetric positive definite $n\times n$ matrix $A$ such that $A^2=C$.
  2. Generate (simulate) $n$ independent standard normal random variables $Y_1,\dotsc, Y_n$. Denote by $Y$ the random vector $(Y_1,\dotsc, Y_n)$.
  3. The random vector $AY$ is Gaussian with covariance matrix $C$.
  • $\begingroup$ Thank you very much for your answer. but sorry in point 2, what do you exactly mean by normal? $\endgroup$ – Niro Nov 13 '16 at 14:47
  • $\begingroup$ A normal random variable is a Gaussian random variable with mean $0$, variance $1$ and hence distribution $(2\pi)^{-\frac{1}{2}} e^{-\frac{x^2}{2}}dx$. $\endgroup$ – Liviu Nicolaescu Nov 13 '16 at 16:13

Since this is a "spherical" Gaussian Distribution there is no interaction between the dimensions. This can also be seen from the fact that the second parameter of N contains the identity matrix, so all the off-diagonal values (corresponding to correlations between dimensions) are 0. (I do not understand the meaning of parameter lambda here though) In that case, if the Gaussian distribution is really spherical, you can just sample each dimension separately for each vector.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.