What books provide an overview of computational statistics as it applies to computer science?

Question

As a software engineer, I'm interested in topics such as statistical algorithms, data mining, machine learning, Bayesian networks, classification algorithms, neural networks, Markov chains, Monte Carlo methods, and random number generation.

I personally haven't had the pleasure of working hands-on with any of these techniques, but I have had to work with software that, under the hood, employed them and would like to know more about them, at a high level. I'm looking for books that cover a great breadth - great depth is not necessary at this point. I think that I can learn a lot about software development if I can understand the mathematical foundations behind the algorithms and techniques that are employed.

Can the Statistical Analysis community recommend books that I can use to learn more about implementing various statistical elements in software?

Answer 1

I'd suggest Christopher Bishop's "Pattern Recognition and Machine Learning". You can see some of it, including a sample chapter, at https://www.microsoft.com/en-us/research/people/cmbishop/#!prml-book

Answer 2

You might want to read the extremely popular question on Stack Overflow on what statistics a programmer or computer scientist should know.

Answer 3

Here is a very nice book from James E. Gentle, Computational Statistics (Springer, 2009), which covers both computational and statistical aspects of data analysis. Gentle also authored other great books, check his publications.

Another great book is the Handbook of Computational Statistics, from Gentle et al. (Springer, 2004); it is circulating as PDF somewhere on the web, so just try looking at it on Google.

Answer 4

You've mentioned some ML techniques, so two quite nice books (quite because unfortunately my favorite is in Polish):
http://www.amazon.com/Machine-Learning-Algorithmic-Perspective-Recognition/dp/1420067184
http://ai.stanford.edu/~nilsson/mlbook.html

For numeric stuff like random number generation:
http://www.nr.com/

Answer 5

I picked up a copy of Probability and Statistics for Computer Scientists - Michael Baron on sale with another statistics book (I honestly bought it because of the name - I wanted a book that would take some kind of look at statistics from a computer science perspective, even if it wasn't perfect). I haven't had a chance to read it or work any problems in it yet, but it seems like a solid book.

The preface of the book says that it's for upper level undergraduate students and beginning graduate students, and I would agree with this. Some understanding of probability and statistics are necessary to grasp the contents of this book.

Topics include probability, discrete random variables, continuous distributions, Monte Carlo methods, stochastic processes, queuing systems, statistical inference, and regression.

Answer 6

Although it's not specifically computational statistics, A Handbook of Statistical Analyses Using R - Brian S. Everitt and Torsten Hothorn covers a lot of topics that I've seen covered in basic and intermediate statistics books - inference, ANOVA, linear regression, logistic regression, density estimation, recursive partitioning, principal component analysis, and cluster analysis - using the R language. This might be of interest to those interested in programming.

However, unlike other books, the emphasis is on using the R language to carry out these statistical functions. Other books I've seen use combinations of algebra and calculus to demonstrate statistics. This book actually focuses on how to analyze data using the R language. And to make it even more useful, the data sets the authors use are in CRAN - the R Repository.

Answer 7

Statistical Computing with R - Maria L. Rizzo covers a lot of the topics in Probability and Statistics for Computer Scientists - basic probability and statistics, random variables, Bayesian statistics, Markov chains, visualization of multivariate data, Monte Carlo methods, Permutation tests, probability density estimation, and numerical methods.

The equations and formulas used are presented both as mathematical formulas as well as in R code. I would say that a basic knowledge of probability, statistics, calculus, and maybe discrete mathematics would be advisable for anyone who wants to read this book. A programming background would also be helpful, but there are some references for the R language, operators, and syntax.

Answer 8

As a computer engineer coming to data analysis myself, a really readable book that covers things from a pretty unintimidating & readable perspective (at the cost of not covering as much as any of the other books suggested here) was Programming Collective Intelligence by Toby Segaran. I found it a lot more approachable than, for example, Bishop's book, which a great reference but goes into more depth that you probably want at a first pass. On amazon: http://www.amazon.com/Programming-Collective-Intelligence-Building-Applications/dp/0596529325

Answer 9

CRAN has several good examples of books pertaining to statistical programming. Some of them will not pertain to machine learning and MCMC, but each entry is annotated, so you should have a rough idea of what each book contains to dive a bit further. http://www.r-project.org/doc/bib/R-books.html