Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

As a software engineer, I'm interested in topics such as statistical algorithms, data mining, machine learning, Bayesian networks, classification algorithms, neural networks, Markov chains, Monte Carlo methods, and random number generation.

I personally haven't had the pleasure of working hands-on with any of these techniques, but I have had to work with software that, under the hood, employed them and would like to know more about them, at a high level. I'm looking for books that cover a great breadth - great depth is not necessary at this point. I think that I can learn a lot about software development if I can understand the mathematical foundations behind the algorithms and techniques that are employed.

Can the Statistical Analysis community recommend books that I can use to learn more about implementing various statistical elements in software?

share|improve this question

I'd suggest Christopher Bishop's "Pattern Recognition and Machine Learning". You can see some of it, including a sample chapter, at

share|improve this answer

Here is a very nice book from James E. Gentle, Computational Statistics (Springer, 2009), which covers both computational and statistical aspects of data analysis. Gentle also authored other great books, check his publications.

Another great book is the Handbook of Computational Statistics, from Gentle et al. (Springer, 2004); it is circulating as PDF somewhere on the web, so just try looking at it on Google.

share|improve this answer

You might want to read the extremely popular question on Stack Overflow on what statistics a programmer or computer scientist should know.

share|improve this answer
Good call. I use SO and I was unaware that question even existed. It's good to have a cross-linking for times when questions might appear on multiple exchanges. – Thomas Owens Aug 14 '10 at 1:46

You've mentioned some ML techniques, so two quite nice books (quite because unfortunately my favorite is in Polish):

For numeric stuff like random number generation:

share|improve this answer
+1 the Marsland book is quite good and filled a big gap in the existing selection of ML books. – ars Aug 16 '10 at 3:11

I picked up a copy of Probability and Statistics for Computer Scientists - Michael Baron on sale with another statistics book (I honestly bought it because of the name - I wanted a book that would take some kind of look at statistics from a computer science perspective, even if it wasn't perfect). I haven't had a chance to read it or work any problems in it yet, but it seems like a solid book.

The preface of the book says that it's for upper level undergraduate students and beginning graduate students, and I would agree with this. Some understanding of probability and statistics are necessary to grasp the contents of this book.

Topics include probability, discrete random variables, continuous distributions, Monte Carlo methods, stochastic processes, queuing systems, statistical inference, and regression.

share|improve this answer

Although it's not specifically computational statistics, A Handbook of Statistical Analyses Using R - Brian S. Everitt and Torsten Hothorn covers a lot of topics that I've seen covered in basic and intermediate statistics books - inference, ANOVA, linear regression, logistic regression, density estimation, recursive partitioning, principal component analysis, and cluster analysis - using the R language. This might be of interest to those interested in programming.

However, unlike other books, the emphasis is on using the R language to carry out these statistical functions. Other books I've seen use combinations of algebra and calculus to demonstrate statistics. This book actually focuses on how to analyze data using the R language. And to make it even more useful, the data sets the authors use are in CRAN - the R Repository.

share|improve this answer

Statistical Computing with R - Maria L. Rizzo covers a lot of the topics in Probability and Statistics for Computer Scientists - basic probability and statistics, random variables, Bayesian statistics, Markov chains, visualization of multivariate data, Monte Carlo methods, Permutation tests, probability density estimation, and numerical methods.

The equations and formulas used are presented both as mathematical formulas as well as in R code. I would say that a basic knowledge of probability, statistics, calculus, and maybe discrete mathematics would be advisable for anyone who wants to read this book. A programming background would also be helpful, but there are some references for the R language, operators, and syntax.

share|improve this answer

As a computer engineer coming to data analysis myself, a really readable book that covers things from a pretty unintimidating & readable perspective (at the cost of not covering as much as any of the other books suggested here) was Programming Collective Intelligence by Toby Segaran. I found it a lot more approachable than, for example, Bishop's book, which a great reference but goes into more depth that you probably want at a first pass. On amazon:

share|improve this answer

CRAN has several good examples of books pertaining to statistical programming. Some of them will not pertain to machine learning and MCMC, but each entry is annotated, so you should have a rough idea of what each book contains to dive a bit further.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.