5
$\begingroup$

I have been trying to derive some conditional distribution for parameters of a linear transformation (represented as a matrix) and I had a lot of help on this thread yesterday. However, I realised I did something which could be a terrible mistake.

So, I have vector valued observation $y$, which is modelled as a linear combination of another observation $x$.

The way $y$ is modelled is:

$$ y \sim \mathrm{N} (Ax, \Sigma). $$

The thing to note here is that $A$ is a matrix. Now, I wanted to put a normal prior on the transformation parameters i.e. the entries of $A$ and I do that as:

$$ A \sim \mathrm{N} (A_0, \nabla). $$

Now, here I see $A$ as a vector. Now, when I tried to get the conditional distribution of $A$ by multiplying the two Gaussians, I ran into a bit of trouble because of this matrix-vector discrepancy. I could not separate the terms properly as was suggested by Glen_b in that thread (because my question had the mistake and I did not realize that then).

I was wondering if there is a way to deal with this so that I can still derive the conditional distribution in a closed form way. Perhaps, what I have done is valid and I need to find some linear algebra tricks to make this work. However, I think that might not be the case.

On a more hopeful note, I see that there is this Matrix Normal Distribution Would it be possible to use this as a prior for the transformation matrix $A$ and would it still be possible to get a closed form solution for the conditional posterior? I am sure this entity is a lot more complex to manipulate but perhaps someone with more expertise can confirm if this is a good road to go down.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

If you observe $Ax = (x^\top \otimes I_m) vec(A)$, then you can put a prior over $A$ (matrix-normal prior is on $vec(A)$), so you can do proper Gaussian inference, in closed form. As far as I am aware, you need to put a special structure on prior covariance matrix to make the inference tractable. Regarding posterior updates, you can find exactly what you are looking for, from here. (Sec. II-B, Inference section). Note the special structure on the covariance matrix.

Improve this answer
$\endgroup$
8
  • $\begingroup$ One thing I am confused about is why is the matrix-normal prior on $vec(A)$ rather than $A$ itself? $\endgroup$
    – Luca
    Commented Nov 4, 2015 at 14:34
  • 1
    $\begingroup$ My motivation comes from converting $Ax$ to some model over $A$. It seemed easy to do so. I am not sure what will happen if you derive everything in directly matrix form. If you do, please post here the derivation! :) $\endgroup$
    – user16776
    Commented Nov 4, 2015 at 15:05
  • 1
    $\begingroup$ I am going through the paper now and will update the thread on my progress! Since, we are putting a prior over $vec(A)$, is it not enough to use a MVN prior rather than a matrix-normal? $\endgroup$
    – Luca
    Commented Nov 4, 2015 at 15:08
  • 1
    $\begingroup$ Assuming a special covariance structure on MVN prior on $vec(A)$ enabled me to go back to the matrix form (and you see in the Wikipedia, it also corresponds to assume a Matrix-normal prior with $U=I$). If you assume a MVN prior with full covariance on $vec(A)$, I suspect posterior is not tractable in matrix-form. I wanted to avoid vectorising and de-vectorising after each observation, and wanted to keep matrix-form along updates. (I will try to obtain a direct matrix-form as well!). $\endgroup$
    – user16776
    Commented Nov 4, 2015 at 15:18
  • 1
    $\begingroup$ Yes it seems so. But if you assume $vec(A) \sim \mathcal{N}(\mu, sI)$ with $s$ being some number, and $I$ is the identity matrix -- I don't know if there is a way to specify $V$ and $U$ (they will be constant times identity, but constants are not easy to say I guess). $\endgroup$
    – user16776
    Commented Nov 6, 2015 at 23:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.