# Learning material for regression analysis

I found a lecture notes from regression analysis but it was quite hard to learn from it. Those notes were aimed for students who had read just basics of statistics beforehand. I have a background in mathematics, read basic course of probability and measure theory/real analysis but wrote my master's thesis from algebra so I can prove theorems on my own and fill the details of reasoning.

But I would like to learn on my own what kind of methods I can use in particular situations and to improve my skills to have a job. I also like if there is some explanation why those methods works in particular situations. Those lecture notes had the following subjects:

One and two way anova, one and several explanations linear regression model, logistic regression and applying regression model to analysing the trend and seasonality.

What would one suggest to learn the topic on my own?

• The way this question is asked suggests you are interested in the mathematical underpinnings of "regression analysis" (which is a broad, vague topic by the way) rather than in regression itself. Could you clarify this point? When you do, could you also indicate a little more precisely what you mean by "regression analysis"? – whuber Aug 19 '15 at 12:55
• Could you tell a little bit more detailed about your background in maths? Did you study probability? You could have a look at math.stackexchange.com/questions/735075/… – kjetil b halvorsen Aug 19 '15 at 13:14

For the basics I tend to turn to the online "Hanbook of biological statistics" The chapter on linear regression is quite simple to grasp. If you are looking for more in depth information I enjoy "Biostatistical analysis" by Zar.

Linear regression is used in many fields. The idea is to fit a trend line to a dataset so you may predict a response variable given certain independent variables.

Say if you had some data of how many hours students study per night and their corresponding GPAs. You could use regression to predict your GPA based on how many hours you study per night. This is a simplified example and there are obviously more variables that affect GPA; so to get the most accurate prediction you would include all the variables that affect GPA.

As a math major, I would start by reading up on Ordinary Least Squares Method.

I suggest entering your list of topics in YouTube. I have found the videos there to be very useful in learning the techniques, although more so for the general public as opposed to a Math major. One example: https://www.youtube.com/watch?v=aq8VU5KLmkY

I would recommend you the Data Analysis Using Regression and Multilevel/Hierarchical Models book by Andrew Gelman and Jennifer Hill. It covers a broad variety of topics in regression analysis, including hierarchical and Bayesian models. It is very readable and has many code examples. It doesn't go in-depth into theory, rather focuses on practical applications while helping you to understand the methods, but rather focuses on building intuition, than on formal proofs.

For a more sophisticated approach to statistical inference, I would recommend Casella and Berger, Statistical Inference. You can skip the early chapters on probability if you are already familiar with measure-theoretic probability and move on to the later chapters that deal with the topics you mentioned. In particular, chapters on Hypothesis Testing, Asymptotics, and Regression Models should be useful. If you want an even more advanced reference (graduate level) see Lehman and Romano, Testing Statistical Hypotheses and Lehman, Theory of Point Estimation. These last two are significantly more mathematically sophisticated and assume you are comfortable with at least Real Analysis (which it seems that you are).

Since you mention applying linear regression I would also recommend one of the classic books on econometrics. My favorites are Wooldridge, Econometric Analysis of Cross Section and Panel Data and Bruce Hansen's Econometrics. This last one is free online and he continues to keep it updated which is wonderful.