71
$\begingroup$

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: maximum likelihood, generalized linear models, principal component analysis, non-linear models. I've tried Statistical Models by A.C. Davison but frankly I had to put it down after 2 chapters. The text is encyclopedic in its coverage and mathematical treats but, as a practitioner, I like to approach subjects by understanding the intuition first, and then delve into the mathematical background.

These are some texts that I consider outstanding for their pedagogical value. I would like to find an equivalent for the more advanced subjects I mentioned.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ I wonder... how does the Hyndman et al. book treat the topics you list above? The normal treatment of these issues in forecasting is quite specific to the field, so I would not expect one to learn a lot for application to general statistics from a forecasting book. $\endgroup$
    – Stephan Kolassa
    Commented Jul 27, 2012 at 20:28
  • $\begingroup$ @StephanKolassa The books I listed are just examples of introductory statistics that I mentioned for the pedagogical value. $\endgroup$
    – Robert Kubrick
    Commented Jul 27, 2012 at 20:54

9 Answers 9

Reset to default
25
$\begingroup$

Maximum likelihood: In all Likelihood (Pawitan). Moderately clear book and the most clear (IMO) with respect to books dealing with likelihood only. Also has R code.

GLMs: Categorical Data Analysis (Agresti, 2002) is one of the best written stat books I have read (also has R code available). This text will also help with maximum likelihood. The third edition is coming out in a few months.

Second on my list for the above two is Collett's Modelling Binary Data.

PCA: I find Rencher's writing clear in Methods of multivariate analysis. This is a graduate level text, but it is introductory.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ I agree. I may be prejudice because I think he references my bootstrap bppk and not many people do. $\endgroup$
    – Michael R. Chernick
    Commented Jul 28, 2012 at 1:45
  • $\begingroup$ Your books are great. :) If the question had asked for a bootstrap book, they would have been on my list. $\endgroup$
    – julieth
    Commented Jul 28, 2012 at 1:53
  • 2
    $\begingroup$ +1 for Collet's book. It contains lots of useful data sets. $\endgroup$
    – user10525
    Commented Jul 28, 2012 at 15:31
  • $\begingroup$ Thanks to all for the excellent selection. I chose this answer simply because is short in recs and offers more breadth in terms of subjects treated (PCA, likelihood, multivariate analysis). I will only know which books are best after reading them, of course. Agresti is not included here but it has strong support by the other posters. $\endgroup$
    – Robert Kubrick
    Commented Jul 28, 2012 at 19:32
23
$\begingroup$

Some books on Likelihood Estimation

  • * Amari, Barndorff-Nielsen, Kass, Lauritzen and Rao, Differential geometry in statistical inference. $-\small{\text{Geometrical approach for proving existence, uniqueness and other properties of MLE.}}$

  • * Butler, Saddlepoint Approximations with Applications.
    $-\small{\text{Saddlepoint approximations to the MLE on complicated models.}}$

  • * Cox, Principles of Statistical Inference.
    $-\small{\text{A basic reference on MLE.}}$

  • * Cox and Barndorff-Nielsen, Inference and Asymptotics. $-\small{\text{Likelihood, pseudo-likelihood, approximation theorems and asymptotics explained by}}$ $ \small{\text{two exponents in this area.}}$

  • * Edwards, Likelihood.
    $-\small{\text{A reference for a general discussion on this concept.}}$

  • * Ferguson, A Course in Large Sample Theory. $-\small{\text{Contains classical results on asymptotic properties of point estimators.}}$

  • * Kalbfleisch, Probability and Statistical Inference II. $\spadesuit$
    $-\small{\text{Introductory book containing interesting basic results such as the continuous }}$ $\small{\text{approximation to the likelihood which is not always explained.}}$

  • * Lehmann and Casella, Theory of Point Estimation.
    $-\small{\text{Classical results on point estimation, an essential reference.}}$

  • * Pace and Salvan, Principles of Statistical Inference: From a Neo-Fisherian Perspective. $-\small{\text{A good reference on a school of thought becoming more and more popular:}}$ $\small{\text{the Neo-Fisherian.}}$

  • * Pawitan, In All Likelihood: Statistical Modelling and Inference Using Likelihood.

  • * Serfling, Approximation Theorems of Mathematical Statistics. $-\small{\text{More rigorous book, here you can find the mystical "regularity conditions".}}$

  • * Severini, Likelihood Methods in Statistics.

  • * Shao, Mathematical Statistics.
    $-\small{\text{Classical results, good as a textbook.}}$

  • * Sprott, Statistical Inference in Science. $\spadesuit$ $-\small{\text{Basic reference on likelihood, profile likelihood and classical statistical modelling.}}$

  • * van der Vaart, Asymptotic Statistics.
    $-\small{\text{A general reference on: modes of convergence, properties of MLE, delta method,}}$ $\small{\text{ moment estimators, efficiency and tests.}}$

  • * Young and Smith, Essentials of Statistical Inference. $-\small{\text{A more recent book on: Likelihood, pseudolikelihood, saddlepoint approximations,}}$ $\small{p^*\text{ formula, modified profile likelihoods and more.}}$

$\spadesuit$ Suggestion for the OP

Improve this answer
$\endgroup$
12
  • $\begingroup$ Are these listed in a particular order (e.g., your favorite to your least favorite) or not? $\endgroup$
    – Jake Westfall
    Commented Jul 27, 2012 at 16:46
  • $\begingroup$ @Jake As my memory brought them back. I will include more as soon as I manage to recall them and then I will organise them in alphabetical order. $\endgroup$
    – user10525
    Commented Jul 27, 2012 at 16:48
  • $\begingroup$ @Procrastinator Thanks for the exhaustive list, but I am more interested in specific recommendations focused on the criteria I described rather than a large list. $\endgroup$
    – Robert Kubrick
    Commented Jul 27, 2012 at 19:02
  • $\begingroup$ @RobertKubrick I do not quite understand your comment, but of course, it is a matter of taste. I am focused on books dealing with some aspects of likelihood estimation. I can tell you some particular aspects if you wish. This topic is too extensive to be summarised in a couple of books ... and believe me, I am not choosing books randomly. $\endgroup$
    – user10525
    Commented Jul 27, 2012 at 19:06
  • 4
    $\begingroup$ I am familiar with some of the books and for the ones I know I think Procrastinator has identified books that fit the criteria. But Robert Kubrick what do you expect of us. This is a difficult question and we serve you best by giving you a good list. in the end you have personal choices to make and we don't know you well enough to pick for you. We are not saying buy every book on the list . But you can go to amazon and take a peek inside. Read the customer reviews and the publisher descriptions. $\endgroup$
    – Michael R. Chernick
    Commented Jul 27, 2012 at 23:21
13
$\begingroup$

My guess is that, for your requirements, the best book on generalized linear models is probably:

There are other books that might be considered better, but I suspect would be less appealing to a practitioner who would prefer to avoid dense mathematics:

As for your other topics, I'm afraid I don't know of books for them, but maybe others can make some recommendations.

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ McCullagh & Nelder certainly demands some mathematical sophistication, but I think "completely impenetrable for all but very advanced mathematical statisticians" overdoes it. I think it's less mathematically demanding than, say, Hogg & Craik. $\endgroup$
    – Peter Flom
    Commented Jul 27, 2012 at 17:40
  • $\begingroup$ Sorry about the hyperbole, @Peter, I've edited the comment. (However, that was in line w/ what I've heard; I note that I haven't actually read it.) $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 27, 2012 at 17:55
  • $\begingroup$ Good suggestions @Gung. $\endgroup$
    – Michael R. Chernick
    Commented Jul 27, 2012 at 23:23
  • $\begingroup$ Peter did you mean Hogg & Craig. Bob Hogg has a new coauthor in the recent editins Elliot Tanis. $\endgroup$
    – Michael R. Chernick
    Commented Jul 27, 2012 at 23:26
  • 2
    $\begingroup$ I would recommend anything by Agresti. He's very high on the clarity/sophistication index. That is, at any given level of mathematical sophistication, Agresti writes clearly, compared to others. $\endgroup$
    – Peter Flom
    Commented Jul 28, 2012 at 1:34
8
$\begingroup$

Not sure if these are at the level you're looking for, but some books I've found useful-

GLMs - McCullagh and Nelder is the canonical book

PCA - A User's Guide to Principal Components - despite the title it does go into some degree of depth on the topic

Improve this answer
$\endgroup$
8
$\begingroup$

I really like Larry Wasserman's books "All of Statistics" and "All of Nonparametric Statistics". They are very readable, and cover a lot of ground quickly.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ (+1) I had not realized these (well-received) books included advanced topics as well as introductory material. $\endgroup$
    – whuber ♦
    Commented May 28, 2014 at 17:08
  • $\begingroup$ The only problem is that the book is too brief. $\endgroup$
    – LaTeXFan
    Commented Dec 12, 2014 at 1:53
7
$\begingroup$

The Nonlinear Models books that I like and rely on are (1) Bates and Watts and (2) Gallant. Both are published by Wiley.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Oops, @Gung and I were editing at the same time to fix the links. Now neither edit is there! OK, now I think it's right $\endgroup$
    – Peter Flom
    Commented Jul 28, 2012 at 1:30
  • 1
    $\begingroup$ +1, thanks Michael. I'm sure that will be helpful from the OP. BTW, the way I do links is to copy the link on the other page, double-click / highlight the word or phrase that I want to serve as the hyperlink here, & then click on the button above the text window next to the quotation mark that looks like 3 links from a chain. That opens a wizard where you can paste the web address. Cheers $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 28, 2012 at 1:34
  • $\begingroup$ I'm sure the book is good but $202! $\endgroup$
    – Glen
    Commented Jul 28, 2012 at 13:37
  • 1
    $\begingroup$ Wiley books are a little more expensive than other publishers but this is not far from the going rate for advanced statistics books. I got mine long ago when they were much cheaper. But the answer is then to borrow it from a technical library or get it used. Used copies of texts like these often sell for considerably less evenin good condition, much like buying a used car. $\endgroup$
    – Michael R. Chernick
    Commented Jul 28, 2012 at 19:54
4
$\begingroup$

For Bayesian analysis (including imprecise analysis), I'm going to put in big plugs for:

That last book, by the brilliant Peter Walley, is an eye-opener on different ways of doing sensitivity analysis, and the fact that this can be built into probability theory at an axiomatic level.

Improve this answer
$\endgroup$
3
$\begingroup$

Mehta (2014) Statistical Topics (ISBN: 978-1499273533) is good intermediate level statistics story telling. Doesn't cover much of you topics you noted above though.

Improve this answer
$\endgroup$
1
$\begingroup$

One really simple introductory statistics book is Andy Field's "Discovering Statistics using R" - also available for SPSS. It contains a lot of nice examples and is even fun to read. Less precise, though compared to other books, but with very little mathematical formulations and lots of text. I found it easy for a basic start, and am still using it from time to time.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ That's a reasonable suggestion, but I think it isn't quite what the OP is asking for. $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 2, 2013 at 19:50
  • 1
    $\begingroup$ +1. Andy Field's books are not at all what this question is about so far as I can tell. $\endgroup$
    – Nick Cox
    Commented Jun 9, 2014 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.