I just started a new job essentially working as a Data Scientist / Software Engineer. I have a background as a math Ph.D. and as part of that training, I've taken the typical measure theoretic probability class and fair amount of analysis (measure theory, Fourier, PDEs). However, I've never really studied any practical probability theory nor stats and one gap is all this stuff about random vectors, covariance matrices their significance etc. I can look this stuff up on Wikipedia, but a coherent treatment would be nice, so I don't end up missing things one should know about.
Question: Does anyone have a recommendation of a good book that talks about joint distributions, multivariate Gaussians etc. and whatever other important distributions there are in the higher-dimensional settings? Preferably something that assumes that the reader is familiar with linear algebra. It also wouldn't hurt if the book contained rigorous treatments of stuff like SVD and how it applies to PCA, since this is not typically part of a pure math linear algebra class. I assume there are other similar topics in linear algebra, which are useful in stats, but typically omitted in the standard linear algebra and grad abstract algebra classes. If no single book covers this, a reading list of a few books would not hurt either.