A good book for regression analysis for pure mathematicians I am looking for a good book for regression that is mainly focused on mathematical stuff instead of applied parts. Basically, I want a book that contains all the proofs and mathematical descriptions related to regression analysis in a purely mathematical way.
I looked for recommendations given for other related quarries but I couldn't find any good book dedicated to regression in a pure mathematical way, I mean contains all the proofs and mathematical relations between different concepts.
These are some topics I am mainly interested in :

*

*Simple Linear Regression (Review estimation)

*Test of the parameters

*Validation of the model and its assumptions ($R^2$ value, test of the model,
residual analysis)

*Multivariate Normal and related results

*Multiple Linear Regression

*Estimation of the parameters

*Projection revisited

*Tests for the parameters

*Validation of the model and its assumptions ($R^2$, adjusted-$R^2$, test of the
model, residual analysis)

*Variable selection (forward, backward, AIC)

*Multicollinearity (Ridge-regression)

Note: I know it might be possible that a single book does not cover all topics in detail but I am fine with reading a different book for different topics.
 A: I would recommend Seber & Lee (from which I originally learned regression.) Cover most of your topics with proofs.  An alternative in the same style, but also covering glm's is Linear Models and Generalizations : Least Squares and Alternatives by Rao et al.
A shorter book with a more geometric viewpoint is The Coordinate-Free Approach to Linear Models by  Michael J. Wichura, but it will not cover all your topics.
A: I think that you might be interested in
Stachurski (2016) A Primer in Econometric Theory.
The book is quite mathematically oriented. The book is organized into 3 sections:

*

*Background - which is all about pure mathematical foundations; vector spaces, linear algebra and matrices, foundations of probability, modeling dependence, asymptotics etc.


*Foundations of statistics - which covers rigorous mathematical definition of many basic statistical concepts, properties of estimators, confidence sets etc.


*Econometric models: this section covers some econometric models in great detail. It’s range of models is not wide but it goes quite in depth for each topic (there is whole chapter just on geometry of least squares).
I think based on your description this is what you are looking for. It definitely covers most of the topics you listed (but not all I don’t think information criteria like AIC (they are mentioned but not discussed in great detail) but otherwise everything you listed should be covered). It is very in depth and focused on theory rather than practical applications.
