# Texts on Various Topics in Statistics (GLMs, MCMC, Decision Trees, etc.)

I am currently looking for texts (or preferably a specific text) which have a good balance between theory and application and are as comprehensive as possible and are at an introductory level, covering these topics:

Markov Chains, MCMC, GLMs and extended linear models, Decision Trees, Spatial Statistics, Visualization & Fit, Neural Nets, Ensemble Methods

From my research, it seems that these are topics covered in a master's level statistics program, but I've only had an undergraduate background and don't know what are the standard texts for such topics.

Edit: Needless to say, I would like more detailed answers/feedback on this topic.

Edit2: The answers below are nice, but I would like answers which address all topics if possible. As mentioned in the bounty description, this list of topics will be covered in a future actuarial exam. No source material has been announced for this exam yet.

• I would personally suggest "Elements of Statistical Learning" – user53346 Aug 4 '14 at 5:46
• For Visualization & Fit, the best educational sources I currently know of, is whuber and Glen_b. You can google them. – Alecos Papadopoulos Aug 18 '14 at 2:21
• You're looking for a book with all of these in one place? I'm not sure that exists unless you consider Wikipedia a book. – shadowtalker Aug 22 '14 at 2:15
• Note that I said "texts." The more compact, the better. – Clarinetist Aug 22 '14 at 4:26

(Update 24-8-2014: Added things on Markov Chains).

I doubt that there is a single "textbook" that covers all this. So partial suggestions for just three of these topics:

For Markov Chains, I provide first two math.SE links that contain many suggestions: https://math.stackexchange.com/questions/15431/good-introductory-book-for-markov-processes

and

I was able to refresh some of these texts, and they confirmed my impression that Markov Chain books are always much more advanced mathematics than anything else (sometimes, not even the Introduction contains more words than mathematical symbols).

It appears that the most accessible (relatively speaking) is "Markov Chains and Mixing Times" (2008) by D.A. Levin, Y. Peres, and El.L.Wilmer, which is also officially available for download (with a separate file with Errata). At least it contains a picture with a frog in the first page of the first chapter, and the general title of Part II of the book is "The Plot Thickens" -so we have been given fair warning.

For MarkovChainMonteCarlo, if you type "Introduction to MCMC" in a search engine you will be hit by almost $10$ freely downlodable .pdf files (written by academics and for educational purposes in their respective universities). I would suggest to examine every one of them, and check for level/style that suits you. It does not need necessarily to be a published book.

For GeneralizedLinearModels,

Dobson, A. J. (2002). An introduction to generalized linear models (2nd ed). CRC press.

is, I believe, a good (and tested) choice, and has topics relevant to, and some examples directly related to, the insurance field (shouldn't you also go into Extreme Value Theory though?). From the introduction:

The original purpose of the book was to present a unified theoretical and conceptual framework for statistical modelling in a way that was accessible to undergraduate students and researchers in other fields.

and

There is an emphasis on graphical methods for exploratory data analysis, visualizing numerical optimization (for example, of the likelihood function) and plotting residuals to check the adequacy of models.

Contents

1 Introduction 1.1 Background 1.2 Scope 1.3 Notation 1.4 Distributions related to the Normal distribution 1.5 Quadratic forms 1.6 Estimation 1.7 Exercises

2 Model Fitting 2.1 Introduction 2.2 Examples 2.3 Some principles ofstatistica l modelling 2.4 Notation and coding for explanatory variables 2.5 Exercises 3 Exponential Family and Generalized Linear Models 3.1 Introduction 3.2 Exponential family of distributions 3.3 Properties ofdistribution s in the exponential family 3.4 Generalized linear models 3.5 Examples 3.6 Exercises

4 Estimation 4.1 Introduction 4.2 Example: Failure times for pressure vessels 4.3 Maximum likelihood estimation 4.4 Poisson regression example 4.5 Exercises

5 Inference 5.1 Introduction 5.2 Sampling distribution for score statistics 5.3 Taylor series approximations 5.4 Sampling distribution for maximum likelihood estimators 5.5 Log-likelihood ratio statistic 5.6 Sampling distribution for the deviance 5.7 Hypothesis testing 5.8 Exercises

6 Normal Linear Models 6.1 Introduction 6.2 Basic results 6.3 Multiple linear regression 6.4 Analysis of variance 6.5 Analysis ofc ovariance 6.6 General linear models 6.7 Exercises

7 Binary Variables and Logistic Regression 7.1 Probability distributions 7.2 Generalized linear models 7.3 Dose response models 7.4 General logistic regression model 7.5 Goodness offi t statistics 7.6 Residuals 7.7 Other diagnostics 7.8 Example: Senility and WAIS 7.9 Exercises

8 Nominal and Ordinal Logistic Regression 8.1 Introduction 8.2 Multinomial distribution 8.3 Nominal logistic regression 8.4 Ordinal logistic regression 8.5 General comments 8.6 Exercises

9 Count Data, Poisson Regression and Log-Linear Models 9.1 Introduction 9.2 Poisson regression 9.3 Examples ofco ntingency tables 9.4 Probability models for contingency tables 9.5 Log-linear models 9.6 Inference for log-linear models 9.7 Numerical examples 9.8 Remarks 9.9 Exercises

10 Survival Analysis 10.1 Introduction 10.2 Survivor functions and hazard functions 10.3 Empirical survivor function 10.4 Estimation 10.5 Inference 10.6 Model checking 10.7 Example: remission times 10.8 Exercises

11 Clustered and Longitudinal Data 11.1 Introduction 11.2 Example: Recovery from stroke 11.3 Repeated measures models for Normal data 11.4 Repeated measures models for non-Normal data 11.5 Multilevel models 11.6 Stroke example continued 11.7 Comments 11.8 Exercises

Software
References

Given your interests, for GLMs you might consider De Jong and Heller$^{[1]}$ (if you haven't already read it), and maybe add in the document by Clarke and Thayer (if you haven't already read that).

[1]: De Jong P. and Heller, G.Z. (2008),
Generalized linear modelling for insurance data,
Cambridge University Press

[2]: Clark, D.R. and Thayer, C.A. (2004),
"A Primer on the Exponential Family of Distributions,"
CAS Discussion Paper Program, Casualty Actuarial Society, p117-148
http://www.casact.org/pubs/dpp/dpp04/04dpp117.pdf

I strongly suggest you All of statistics for a basic background. Such as covariance, mean, Test Hypothesis and so on.

More related to machine learning the best book is absolutely The element of statistical learning

If you want to go more in the software engineering implementation of this method for big data problem than a good book could be Mining massive dataset

I think that would be very difficult - impossible to learn well only from these books in your own. This is because the field is too wide.

I suggest you to do some free online courses.

The website Udacity as some good course on statistics and machine learning and there is also a very popular course on Coursera.

Here is a good list to learn the art of probability & statistics. Here is another set to learn monte carlo methods. Note, you are better off getting a good grounding in statistics and probability before moving onto MCMC methods as they can seem fairly advanced at first look.

take a look at this list. It has interesting links: http://datascience.sg/resources/