Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have been working in R for a bit and have been faced with things like PCA, SVD, QR decompositions and many such linear algebra results (when inspecting estimating weighted regressions and such) so I wanted to know if anyone has a recommendation on a good comprehensive linear algebra book which is not too theoretical but is mathematically rigorous and covers all of these such topics.

share|improve this question
up vote 25 down vote accepted

The "big three" that I have used/heard of are:

Gentle, Matrix Algebra: Theory, Computations, and Applications in Statistics. (Amazon link).

Searle, Matrix Algebra Useful for Statistics. (Amazon link).

Harville, Matrix Algebra From a Statistician's Perspective. (Amazon link).

I have used Gentle and Harville and found both to be very helpful and quite manageable.

share|improve this answer
There is also F. A. Graybill, Matrices with Applications in Statistics, 2nd. ed., Duxbury, 2001. It has a wealth of good information and serves as a companion volume to Graybill's linear models text. It is, perhaps, slightly dated in that other texts place more emphasis on the power of the SVD. – cardinal Jan 20 '12 at 14:45

The matrix cookbook:

is a free resource will all sorts of useful identities involving various decompositions, forms of inverses for various commonly encountered matrix structures, formulas for differentiating matrix functions and much more. You'll probably find whatever you're looking for in the matrix cookbook. I've never found any mistakes at all there, but since the matrix cookbook is a free resource, it is not professionally edited, so there could potentially be errors there. But, it is regularly being updated, so I wouldn't worry too much about that.

Although this is a general purpose manual, there is certainly a statistics slant to it, as you will see.

share|improve this answer

Matrix Computations by Golub and Van Loan is the standard reference for matrix computation for many.

share|improve this answer

I've found Advanced Multivariate Statistics with Matrices by Kollo and von Rosen to be very useful when working with multivariate statistics. The first 170 pages are linear algebra. It then goes on to cover multivariate distributions, asymptotics and linear models - all in a rigorous way. It doesn't cover projection methods though.

share|improve this answer

In addition to the three mentioned by @Mike Wierzbicki (all of which I use), another useful one is "Matrix Tricks for Linear Statistical Models" by Puntanen, Styan and Isotalo (2011).

share|improve this answer

You could try "Numerical Methods of Statistics", by John F. Monahan. It assumes that you know linear algebra, but the author's web site provides programs coded in R.

share|improve this answer

Krishnan Namboodiri's Matrix Algebra: An Introduction is a quick, bare-bones way to learn much of the linear algebra you'll need.

You can also try MIT OCW.

share|improve this answer

I have Anton's Elementary Linear Algebra, mainly for the chapters on linear equations and matrices and on determinants (I have the 7th edition).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.