In a Metropolis-Hastings algorithm, I'm drawing a matrix from a proposal. The accepted matrices should constitute draws from a posterior.

If it were just a parameter, I would know how to interpret its trace.

However, now I have a matrix. I'm thinking of just evaluating the accepted draws using a density from a matrix distribution, and then see how the plot varies. However, what should I be looking for?

  • $\begingroup$ I will amend my answer below to account for any additional specifics you can provide, e.g., requirements for the model or the matrix or likelihood/data. $\endgroup$ – Peter Leopold Jul 2 '19 at 3:24

I hope I understand you correctly: you are trying to find a matrix that optimizes a model for a given set of data. I am led to believe that the matrix you are varying is the full statement of the model and so represents all of the parameters that aren't otherwise fixed by some conditional statement. You choose Metropolis Hastings to sample from the posterior of model space, viz., matrix space. It also follows that there is some likelihood function defined for your data that will be maximized by a good choice of your model matrix. But as usual there is some wiggle room, and M-H will indeed sample the posterior of your model space (space of matrices) based on your data and the likelihood, and reveal all of the wiggle room in the process.

It would be sensible to follow the standard M-H MCMC algorithm by making small/incremental steps in matrix space. If your matrix is positive definite, your small steps should not violate requirement. I suggest gently rescaling your matrix by multiplying it by a diagonal matrix with diagonal elements close to and centered around 1. You will also want to rotate your matrix gently by multiplying it by a randomly selected rotation matrix with a small angle centered at zero. Other options, like flipping the sign of an eigenvalue, are bolder and will help you move around in matrix space a lot fast and go a lot farther, but you may lose some essential matrix property that you should try to hold invariant, like the trace. But I really don't know what problem you are trying to solve, so it really depends on those details.

What should you be looking for? An acceptance ratio of 30-50%. Constancy of whatever invariants you hold dear. A familiar-looking distribution of your data's likelihood values, or perhaps the negative log likelihood, which is usually what you calculate and can interpret most directly (as energy/$k_BT$, for instance, in physics.) And you should make sure that the time-course of parameter values (matrix elements, trace, what-not) has an autocorrelation function with a characteristic decorrelation time much, much less than the duration of your simulation. That is, you should just be looking at all the standard diagnostics of M-H MCMC. The fact that your parameter space constitutes a set of matrix elements doesn't change any of this.

  • $\begingroup$ Hi Peter, thanks for the answer. The matrix I'm trying to draw does not represent the full model. The remaining parameters mix well. This matrix is being drawn from an Inverse-Wishart, whose mean/mode is located at previous one plus a matrix taken at random (also from a IW): $\mu_{-1}+s_1*M$, where $s_1\sim \text{Unif}(0,1)$ $\endgroup$ – An old man in the sea. Jul 2 '19 at 9:27
  • $\begingroup$ It seems that you might be modeling a covariance matrix. Covariance matrices are positive semi-definite; the set of positive semi-definite matrices is convex, so you are at liberty to perform the affine transformation you suggest. I'd still set $s_1 << 1$ small rather than random uniform $runif(0,1)$. Otherwise, you'll be jumping all over matrix space. Your acceptance rate will be low & you'll miss the wonderful "thermal equilibrium" sampling feature that M.-H. does best. A small $s_1$ means your selection of $M$ from the Inverse-Wishart can be unrestricted. Let the randomness reside there. $\endgroup$ – Peter Leopold Jul 2 '19 at 14:41
  • $\begingroup$ Yes, it's a covariance matrix. I went back to my programme to check what I did. I misreported. I do fix $s_1$. I do $\mu_{-1}+s_1*M$, where $M\sim \text{IW}$. I still not sure I understood your suggestion for a rotation. Where do you multiply the rotation to $\mu_{-1}$, or $M$? In what direction/angle, randomly (uniformly?) selected? Could you please write in latex a formula? Thanks ;) $\endgroup$ – An old man in the sea. Jul 3 '19 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.