6
$\begingroup$

I am wondering about how to specify multivariate normal distributions for vectors that have undergone a Fourier transform. For instance:

Say we have the mean vector $\boldsymbol{\mu}$ and covariance matrix $\boldsymbol{\Sigma}$ of a multivariate normal given in the not-yet Fourier transformed domain. I can draw a vector sample $\mathbf{x}$ from this distribution,

$$ \mathbf{x} \sim N(\boldsymbol{\mu}, \boldsymbol{\Sigma})$$

I can compute the PDF of my sample $\mathbf{x}$ using $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$. Then, this sample $\mathbf{x}$ is Fourier transformed, $FT(\mathbf{x}) = \mathbf{\tilde x}$.

What steps do I take to transform $N(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ so that I can compute the PDF of the Fourier transformed sample $\mathbf{\tilde x}$ directly?

Improve this question
$\endgroup$
6
  • $\begingroup$ I'm not sure I understand the "scenario" for your question. Does en.wikipedia.org/wiki/… contain the answer you need? I think it might, but as I said, I am not sure what your scenario is. If this is what you want, I can make this into an answer. $\endgroup$
    – Mark L. Stone
    Commented Jul 2, 2015 at 14:55
  • $\begingroup$ I added some clarification, is it clearer now? $\endgroup$
    – bill_e
    Commented Jul 2, 2015 at 15:10
  • 1
    $\begingroup$ The Fourier transform of a Gaussian distribution is, I think, a Gaussian distribution. If it was narrow in time or space then it is wide in frequency or wavenumber. Algebraic solution for some examples might help. $\endgroup$
    – EngrStudent
    Commented Jul 2, 2015 at 15:35
  • $\begingroup$ Yes, intuitively that makes sense. The math to go from time domain to freq domain is where I'm coming up short, especially regarding transforming the covariance matrix. $\endgroup$
    – bill_e
    Commented Jul 2, 2015 at 15:41
  • $\begingroup$ Please state clearly and explicitly what the transformation is for which you don't know what the resulting covariance is. Don't just state "It's simple to just Fourier transform x and μ,". State explicitly the transformation(s) you use, and what random variable you wish to find the covariance of. So you start with a random variable X, which is distributed as a Multivariate Normal with mean μ and covariance Σ. Then you apply a transformation: tell us exactly what the transformation is, e.g., Y=f(X), and you tell us what f is. Then you want to know something, tell us what (covariance?) of Y. $\endgroup$
    – Mark L. Stone
    Commented Jul 2, 2015 at 16:05

2 Answers 2

Reset to default
5
+50
$\begingroup$

The multivariate normal has some nice properties. In particular, if $x\sim N(\mu,\Sigma)$, then, for any matrix $A$, $Ax \sim N(A\mu, A\Sigma A^T)$.

Noting that a (discrete) Fourier transform can an be written in matrix form as $FT(x) = Fx$, we see that $FT(x) \sim N(F\mu, F\Sigma F^T)$.

You can prove this by checking the first and second moments. $E(Fx) = FE(x)$ and similar for $E((x- F\mu)(x-F\mu)^T)$).

This idea is particularly important when $\Sigma$ is a circulant matrix, in which case $F\Sigma F^T$ is diagonal (and hence easier to work with numerically and theoretically).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Unfortunately my $\Sigma$ is not circulant, but interestingly, if I try this with a diagonal $\Sigma$, the result of $W \Sigma W^T$ IS circulant. I'm new to time series methods, but is it true that the diagonal of $W \Sigma W^T$ is the power specral density? $\endgroup$
    – bill_e
    Commented Jul 4, 2015 at 15:14
  • 1
    $\begingroup$ An alternative definition of a circulant matrix is that it can be diagonalised by a discrete Fourier transform. If your matrix isn't circulant (or, with some tricks, Toeplitz), there isn't any computational advantage in using an FFT. $\endgroup$
    – Dan Simpson
    Commented Jul 7, 2015 at 10:01
2
$\begingroup$

Unfortunately, the definition and conventions for complex Multivariate Normal is not completely standardized. Perhaps "yours" is consistent with equatoion 2 of http://cran.r-project.org/web/packages/cmvnorm/vignettes/complicator.pdf , which is different than https://en.wikipedia.org/wiki/Complex_normal_distribution .

You can invert the Fourier transform of a real Multivariate Normal per 0.7 equation 9b of https://www.cs.nyu.edu/~roweis/notes/gaussid.pdf . That will leave you with a real Multivariate normal, plus your existing one; then simulate to your heart's content. Once you have the exact definition and convention used in your complex Multivariate Normal, equation 9b needs to be adapted accordingly. You ought to be able to handle it at that point.

The other way of going is to use 0.7 equation 9a of https://www.cs.nyu.edu/~roweis/notes/gaussid.pdf on x. But no matter which way you go, you're living dangerously if you don't understand the conventions used in your Fourier transform of μ. You seem to be playing fast and loose, as evidenced by your terminology of μ as the mean of a Multivariate Normal, and also as something on which you are applying Fourier transform to get μ~. It doesn't mean anything is wrong, but you can easily mix up things which use different conventions, and wind up with the wrong answer.

Improve this answer
$\endgroup$
11
  • $\begingroup$ Thanks Mark, so essentially it boils down to using the identities in 0.7 equation 9a, 9b (yes being very careful with Fourier conventions...)? Do you happen to know how, or where, these identities are derived? You just take an inverse of the covariance matrix (now it is the precision matrix) and voila you've "Fourier transformed" your covariance matrix? That seems strange... $\endgroup$
    – bill_e
    Commented Jul 2, 2015 at 20:54
  • $\begingroup$ In the first sentence in your 2nd paragraph do you mean "invert the Fourier transform of a real Multivariate Normal" or complex multivariate normal? Also, Yes, you're right, I will edit my question to not be so fast and loose. $\endgroup$
    – bill_e
    Commented Jul 2, 2015 at 20:54
  • $\begingroup$ Look at the end of the first answer of math.stackexchange.com/questions/270566/… . That's the one dimensional version, in which the inverse of the covariance matrix is just the reciprocal of the variance. As I said, you need to be careful on conventions, such as factors of 1/sqrt(2*Pi), which is symmetric in both directions, or 1/(2*Pi) in one direction and 1 in the other direction. You need to be quite clear on exactly what "animal" your complex Multivariate Normal is. But this solves it for you "modulo" the conventions. $\endgroup$
    – Mark L. Stone
    Commented Jul 2, 2015 at 23:22
  • $\begingroup$ The first sentence in the 2nd paragraph of my answer means "invert the Fourier transform of a real Multivariate Normal". Adjustments (pre-processing) would be needed for complex Multivariate Normal, but the adjustments depend on how ti is defined (definition of complex Multivariate Normal plus convention (definition) of Fourier transform).. I leave that as an exercise for you. $\endgroup$
    – Mark L. Stone
    Commented Jul 2, 2015 at 23:25
  • $\begingroup$ Ok, I will work that out this weekend, but I'll accept your answer if you can point me to a citable reference for the 0.7 eq. 9 identity. Where did that person get that identity from? Also, wouldn't the definition of the complex normal in this case just be what is required to specify the pdf for x post-Fourier transform? $\endgroup$
    – bill_e
    Commented Jul 3, 2015 at 0:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.