What are the "best" metrics for covariance matrices, and why? It is clear to me that Frobenius&c are not appropriate, and angle parametrizations have their issues too. Intuitively one might want a compromise between these two, but I would also like to know if there are other aspects to keep in mind and maybe well-established standards.
Common metrics have various drawbacks since they're not natural for covariance matrices, e.g. they often don't especially penalize non PSD matrices or don't behave well w.r.t rank (consider two rotated low-rank covariance ellipsoids: I'd like the same-rank intermediate rotation to have lower distances than the componentwise average, which is not the case with $L_1$ and maybe Frobenius, please correct me here). Also convexity is not always guaranteed. It'd be good seeing these and other issues addressed by a "good" metric.