Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am very interested about the potential of statistical analysis for simulation/forecasting/function estimation, etc.

However, I don't know much about it and my mathematical knowledge is still quite limited -- I am a junior undergraduate student in software engineering.

I am looking for a book that would get me started on certain things which I keep reading about: linear regression and other kinds of regression, bayesian methods, monte carlo methods, machine learning, etc. I also want to get started with R so if there was a book that combined both, that would be awesome.

Preferably, I would like the book to explain things conceptually and not in too much technical details -- I would like statistics to be very intuitive to me, because I understand there are very many risky pitfalls in statistics.

I am off course willing to read more books to improve my understanding of topics which I deem valuable.

share|improve this question
  • Maybe you'd like something like Data Analysis and Graphics Using R: An Example-Based Approach by John Maindonald and W. John Braun

    • Website for book
    • Amazon with assorted reviews
    • I recommend it because the book ticks a few of your boxes; it teaches a little R; it provides an overview of a range of different modelling techniques (e.g., multiple regression, time series, graphics, generalised linear model, etc.) without going into too much mathematical detail; it's fairly applied.
  • I agree with @Greg Snow that you may be better off thinking in terms of reading a number of different books. For each topic you mentioned (e.g., Bayesian statistics, time series, simulations, R, machine learning) there are good books dedicated to that particular topic. You may wish to ask separate questions about what would be a good book given your particular interests in that topic.

  • Good freely available online options

    • Elements of Statistical Learning is an excellent book and is even available online for free. From your post, I get the sense that it might be a little more technical than you want at first, but check it out and see what you think. Maybe you'll be ready for it now; maybe later.
    • Benjamin Bolker's Ecological Models and Data in R is another good one. It is from an ecology perspective, but does explain simulation and model fitting clearly from a relatively non-technical perspective; and it's all implemented in R. You can see all his R code on the website. You can even see the Sweave documents used to generate the book!
    • There's a good list of free R documentation on CRAN with some of the documents also providing broader instruction on statistics.
share|improve this answer

A single book that included all those topics would be pretty impressive, and probably weigh more than you do. That is like asking for a single book that teaches basic programming, C, Java, Perl, and advanced database design all in one book (actually probably more, but I don't know enough software engenering terms to add some more advanced ones in).

Regression itself is usually at least a full college course, Bayesian statistics require a course or 2 of theory before taking the Bayesian course to fully understand, etc.

There is no quick and easy road to what you are trying to do. I would suggest taking some good courses at your university and work from there.

There have been other discussions of good books that you can look through for some ideas.

share|improve this answer
Thank you for your answer. However, I'm not trying to understand everything about everything from one book, but reading say, 50 pages about regression would definitely help me a lot for getting at least some reasonable understanding of this subject... – Jérôme Le Chatelier Jul 15 '11 at 3:39

For a combination of R with many of the methods you describe, in addition to the Maindonald and Braun text mentioned by @Jeromy Anglim, I would suggest you take a look at these two books by Julian Faraway:

Both have reasonably simple introductions to the various topics, the latter covers a vast range of more modern approaches to regression, including many machine learning techniques, but does so at a faster pace with less description, and both exemplify the techniques via R code.

You can get a code off the R Website's Books Section to give you 20% off the RRP if you buy direct from Chapman & Hall/CRC Press, but do check the Amazon price or similar for your region as often the reduction on Amazon is competitive with that of the publisher's price after the discount.

One of the good things about this pair of books is that they give you a good flavour of the modern methods with enough detail to then explore the areas that you want to in further detail with more specialised texts.

Some of the content that went into those books is available in an online PDF by Julian, via the Contributed Documents section of the R Website. I encourage you to browse that section to see if there are other docs that might get you started without you having to shell out any cash. An early version of the text that turned into the first edition of Maindonald and Braun's text can also be found in this section.

share|improve this answer

Well, if you want an overview of most statistical methods, and R code for them, you can't go far wrong with Venables and Ripley's Modern Applied Statistics in S.

Its succint, lucid and has enough R code to get you started on pretty much any statistical topic you care to name.

I bought this book and was wary about the price versus page count, but it was well worth the investment. They do assume calculus and linear algebra, but given that you are an engineer, that shouldn't be too much of a problem.

Their S programming is also wonderful, but probably not what you are looking for right now.

share|improve this answer

Elements of statistical learning can be little intimidating for beginners. I would recommend reading "Introduction to Statistical Learning with Applications in R", which can be downloaded for free from here -> It also has worked out examples in R at the end of every chapter.

"Machine Learning: An algorithmic Perspective" by Stephen Marsland also covers broader range of topics without going too much into maths.

share|improve this answer

The previous answers have a lot on the application side of things. As far as conceptual material goes and good statistical thinking, I would recommend Probability Theory: The logic of science by Edwin Jaynes. The first three chapters are available for free here

It doesn't have a great deal in the way of computer programs though, so the application side of things is on the more stylised problems. Has a brilliant chapter on paradoxes of probability theory, with one exception, the "marginalisation paradox", which is correctly resolved here (although Jaynes essentially "gets the lesson" in that an improper prior should be a limit of a sequence of proper priors).

share|improve this answer
I loved this book myself, but I am not sure it is the place to start trying to build an intuition for statistics. It is a rather polemical and idiosyncratic text. – Ben Lauderdale Jul 16 '11 at 21:32

I would recommend "Time Series Analysis and its applications with R examples" by Shumway and Stoffer

The third edition:

Click and buy

share|improve this answer

The R Cookbook is a great way to jump into R and start learning how to use it. It's very practical, so it's great for learning to use the language, but you should look for a good theory book as well.

share|improve this answer

The suggestions made so far are all excellent but are focused on the most advanced and sophisticated techniques using R software. For an excellent and intuitive overview of classic multivariate techniques, the underlying framework for the most up-to-date approaches, including regression, ANOVA, factor analysis, cluster analysis, discriminant analysis, contingency table analysis and structural equation analysis, Dillon and Goldstein's Multivariate Statistics published by Wiley in the 80s remains a classic. It's lucid and applied in its examples without being overly theoretical or wedded to software.

Dillon and Goldstein is the book I would recommend to anyone who wants an understanding of where modern machine learning methods originated.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.