# Textbooks on linear regression with least squares

I watched several videos on linear regression, mainly from Khan Academy. As I have no background in statistics, I thought this was a good way to get an idea of the topic. However I'm currently writing a bachelor thesis on Structural Equation Modeling and I want to get a deeper understanding of Regression Analysis.

The Book should roughly include these topics:

• linear least squares regression
• variance, covariance

• regression coefficient

• coefficient of determination

• residual analysis (esp. leverage effect)

• multiple regression

I especially enjoyed the videos on khan academy, of the proof of minimizing the squared error to the regression line or illustrations like this one from Wikipedia: I don't like it when books only present a formula and some common rules etc. without any further explanation of these.

F.e. Residuals should be normally distributed.

However I'm also not comfortable with strictly mathematical books, so I'm searching for something in between. A plus would be example code in R, but definitely not a must have.

• You could try elements of statistical learning. It has R examples and you can get the PDF for free – Dan Mar 10 '15 at 12:38
• If you're only just starting to learn it, I don't think a single book would be good advice. You should consider reading more than one book. One of John Fox's books on applied regression might be useful to you. – Glen_b Mar 10 '15 at 13:45
• Don't you have an adviser for your thesis? He should give you books to read. – Aksakal Mar 10 '15 at 13:59
• @Glen_b of course, I'm learning from multiple sources, but I thought it would be nice to have some books to quote from, since I cannot quote a Khan Academy video haha. – jannic Mar 10 '15 at 16:14
• @Aksakal Yes, I do have, but he is not a statistican and his reccommendation concerning SEM literature was already poor in my opinion... – jannic Mar 10 '15 at 16:14

I suggest John Fox's "Applied Regression Analysis and Generalized Linear Models" and its companion text "An R Companion to Applied Regression" for one text on regression. James and Hastie's text is introducing regression to develop ideas for statistical learning. Faraway's text has many insights, but is terse. Gelman's book is also nice, but not my suggestion for an introduction to regression. I suggest Fox's text because it is relatively clear, introductory but covers many topics in depth, and has an R companion.

Practical Regression and Anova using R by Julian Faraway is a good book, and is freely available.

If you happen to read French, I recommend Régression, Théorie et applications by Cornillon and Matzner-Løber.

I like "Data Analysis Using Regression and Multilevel/Hierarchical Models" by Andrew Gelman and Jennifer Hill.

"Extending the Linear Model with R" by Julian Faraway has a great introductory chapter which goes into the general linear model (regression). I've never used "Practical Regression and Anova using R", however a a quick glance it looks good as well!

I have two recommendations.

Since you say you have no background in Statistics I think a good beginning would be: "OpenIntro Statistics" by Diez, Barr, and Çetinkaya-Rundel. It goes from the basic concepts and has an introduction to regression. You can download it here. There is a Data Analysis and Statistical Inference course in Coursera that is based on this book.

After getting the basic concepts, I would then suggest going to "Introduction to Statistical Learning" by James, Witten, Hastie, and Tibshirani. There, regression is treated with more depth. You can also download the source codes in R as well as all the datasets they use. All the material can be downloaded here. There is a Statistical Learning MOOC that covers this book.

I would like to add a book which I randomly found in my university library. It's called Correlation and Regression by Philip Bobko. It is very easy to read and draws bridges between different concepts very elegantly, like f.e. the regression slope and the correlation coefficient.