9
$\begingroup$

I am looking for a good book for regression that is mainly focused on mathematical stuff instead of applied parts. Basically, I want a book that contains all the proofs and mathematical descriptions related to regression analysis in a purely mathematical way.
I looked for recommendations given for other related quarries but I couldn't find any good book dedicated to regression in a pure mathematical way, I mean contains all the proofs and mathematical relations between different concepts.

These are some topics I am mainly interested in :

  • Simple Linear Regression (Review estimation)
  • Test of the parameters
  • Validation of the model and its assumptions ($R^2$ value, test of the model, residual analysis)
  • Multivariate Normal and related results
  • Multiple Linear Regression
  • Estimation of the parameters
  • Projection revisited
  • Tests for the parameters
  • Validation of the model and its assumptions ($R^2$, adjusted-$R^2$, test of the model, residual analysis)
  • Variable selection (forward, backward, AIC)
  • Multicollinearity (Ridge-regression)

Note: I know it might be possible that a single book does not cover all topics in detail but I am fine with reading a different book for different topics.

Improve this question
$\endgroup$
4
  • $\begingroup$ I have already looked into Methods of Multivariate Analysis by Alvin C. Rencher, William F. Christensen ; Elizabeth Peck, Geoffrey Vining, Douglas Montgomery - Introduction to Linear Regression Analysis ;l John Fox's Applied Regression Analysis and Generalized Linear Models ; Friedman, Hastie The Elements of Statistical Learning Methods of Multivariate Analysis by Rencher is a good book for in a way but a lot of topics are missing that I am interested in $\endgroup$
    – IamKnull
    Commented Oct 18, 2020 at 14:43
  • $\begingroup$ You don’t exactly need to work with categories of schemes to understand those topics. Most any graduate-level book is going to hit most of those. What kind of mathematical content do you want? // Harrell’s Regression Modeling Strategies might do what you want. // Statisticians don’t like stepwise regress. Please read what the master has to say: statmodeling.stat.columbia.edu/2014/06/02/…. $\endgroup$
    – Dave
    Commented Oct 18, 2020 at 14:58
  • 1
    $\begingroup$ Hi: This isn't going to hit every topic you mentioned but it's pretty hardcore as far as theorem proof, theorem proof etc. I think that it might be good for some of what you want. books.google.com/books/about/Econometrics.html?id=aXV41gkrsVgC $\endgroup$
    – mlofton
    Commented Oct 18, 2020 at 15:04
  • $\begingroup$ Some related posts: stats.stackexchange.com/questions/38082/… stats.stackexchange.com/questions/414/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 21, 2020 at 16:39

2 Answers 2

Reset to default
8
$\begingroup$

I would recommend Seber & Lee (from which I originally learned regression.) Cover most of your topics with proofs. An alternative in the same style, but also covering glm's is Linear Models and Generalizations : Least Squares and Alternatives by Rao et al.

A shorter book with a more geometric viewpoint is The Coordinate-Free Approach to Linear Models by Michael J. Wichura, but it will not cover all your topics.

Improve this answer
$\endgroup$
4
$\begingroup$

I think that you might be interested in Stachurski (2016) A Primer in Econometric Theory.

The book is quite mathematically oriented. The book is organized into 3 sections:

  1. Background - which is all about pure mathematical foundations; vector spaces, linear algebra and matrices, foundations of probability, modeling dependence, asymptotics etc.

  2. Foundations of statistics - which covers rigorous mathematical definition of many basic statistical concepts, properties of estimators, confidence sets etc.

  3. Econometric models: this section covers some econometric models in great detail. It’s range of models is not wide but it goes quite in depth for each topic (there is whole chapter just on geometry of least squares).

I think based on your description this is what you are looking for. It definitely covers most of the topics you listed (but not all I don’t think information criteria like AIC (they are mentioned but not discussed in great detail) but otherwise everything you listed should be covered). It is very in depth and focused on theory rather than practical applications.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.