I'm trying to work out how you do this from first principles. The Wikipedia page on linear regression gives me enough to solve it with matrix operations through the origin but I can't find much literature on implementing an algorithm for an OLS fit returning coefficients, a t-statistic and an $r^2$ value for the fit

Can anyone point me to good reference?


  • 1
    $\begingroup$ Is the question about understanding the mathematics and/or mathematical formulas behind the mentioned quantities or is it about numerical algorithms to actually compute them? Strangely enough, they're very different questions. $\endgroup$ – cardinal Nov 5 '11 at 21:43
  • $\begingroup$ Both really. I'm primarily in need of implementing it, but I would like to understand the maths properly too. $\endgroup$ – Chris Nov 5 '11 at 22:38
  • $\begingroup$ Let me ask a question so that I understand yours better: Why do you need to implement it yourself? There are (many!) very good reasons not to do so, and instead to rely on available libraries and/or software packages with many tens (or hundreds) of thousands of man-hours already put into them. $\endgroup$ – cardinal Nov 5 '11 at 22:58
  • $\begingroup$ For the mathematical and statistical background of linear regression, any good linear-regression theory text will do. Here is a recent question where three are mentioned. $\endgroup$ – cardinal Nov 5 '11 at 23:03
  • $\begingroup$ Well, one because I feel implementing anything is a great way to understand it properly, and two because I can't find an existing implementation for the platform I require it. $\endgroup$ – Chris Nov 6 '11 at 10:28

There are quite a few books that cover this.

My personal favourite is "Solving least squares problems" by Charles L. Lawson and Richard J. Hanson.

A more recent work is "Numerical methods for least squares problems" By Åke Björck.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.