Are there any free statistical textbooks available?
Online books include
Update: I can now add my own forecasting textbook
There's a superb Probability book here: https://web.archive.org/web/20100102085337/http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html which you can also buy in hardcopy.;
I've often found the Engineering Statistics Handbook useful. It can be found here.
Although I've never read it myself, I hear Introduction to Probability and Statistics Using R is very good. It's a full ~400 page ebook (also available as an actual book). As a bonus, it also teaches you R, which of course you want to learn anyways.
One the most, if not the most, popular textbooks on machine learning is Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, which is fully available online (currently 10th printing). It is comparable in scope e.g. to Bishop's Pattern Recognition and ML or Murphy's ML, but those books are not free, while ESL is.
Hastie & Tibshirani also co-wrote freely available An Introduction to Statistical Learning, With Applications in R which is basically a simpler version of The Elements and focuses on R.
In 2015, Hastie & Tibshirani co-authored a new textbook Statistical Learning with Sparsity: The Lasso and Generalizations, also available online. This one is quite a bit shorter and focuses specifically on lasso.
Another freely available all-encompassing machine learning textbook is David Barber's Bayesian Reasoning and Machine Learning. I did not use it myself, but it is widely considered to be an excellent book.
Switching now to more specialized topics, there are:
Rasmussen & Williams Gaussian Processes for Machine Learning, which is the book on Gaussian processes.
Much awaited Goodfellow, Bengio and Courville Deep Learning textbook that is about to be published by MIT Press. It isn't published yet, but the book is already available online. On the official website one can view it in browser but cannot download (as per agreement with the publisher), but it is easy to find a combined PDF e.g. here on github.
Csaba Szepesvári, Algorithms for Reinforcement Learning, a concise book on RL. A classical, much more detailed but a bit dated textbook is Sutton & Barto, Reinforcement Learning: An Introduction which is also freely available online but only in a cumbersome HTML format.
Boyd and Vandenberghe, Convex Optimization.
I really like The Little Handbook of Statistical Practice by Gerard E. Dallal
Here's a fresh one: Introduction to Probability and Statistics Using R . It's R-specific, though, but it's a great one. I haven't read it yet, but it seems fine so far...
Norman Matloff has written a mathematical statistics textbook for computer science students that's free. Kind of a niche market, I suppose. For what it's worth, I haven't read it, but Matloff has a Ph.D. in mathematical statistics, works for a computer science department, and wrote a really good R book, that I recommend for people who want to go to the next stage of programming R better (as opposed to just fitting models with canned functions).
Inexpensive paperback copies are also available on Amazon.
A New View of Statistics by Will G. Hopkins is great! It is designed to help you understand how to understand the results of statistical analyses, not how to prove statistical theorems.
Not Statistics specific, but a good resource is: http://www.reddit.com/r/mathbooks Also, George Cain at Georgia Tech maintains a list of freely available maths texts that includes some statistical texts. http://people.math.gatech.edu/~cain/textbooks/onlinebooks.html
"An Introduction to Statistical Learning with Applications in R" https://www.statlearning.com/ by two of the 3 authors of the well-known "The Elements of Statistical Learning" plus 2 other authors. An Introduction to Statistical Learning with Applications in R is written at a more introductory level with less mathematical background required than The Elements of Statistical Learning, makes use of R (unlike The Elements of Statistical Learning), and was first published in 2013, some years after this thread was started.
For getting into stochastic processes and SDEs, Tom Kurtz's lecture notes are hard to beat. It starts with a decent review of probability and some convergence results, and then dives right into continuous time stochastic processes in fairly clear, comprehensible language. In general it's one of the best books on the topic -- free or otherwise -- I've found.
I really like these two books by Daniel McFadden of Berkeley:
Lecture Notes: Econometric Tools http://elsa.berkeley.edu/users/mcfadden/e240a_sp98/e240a.html
Lecture Notes: Econometrics/Statistics http://elsa.berkeley.edu/users/mcfadden/e240b_f01/e240b.html
Cosma Shalizi, CMUs ML guru, occasionally updates a draft of a stats book soon to be published by Cambridge Press titled Advanced Data Analysis from an Elementary Point of View. Can't recommend it highly enough...
Here's the Table of contents:
I. Regression and Its Generalizations Regression Basics The Truth about Linear Regression Model Evaluation Smoothing in Regression Simulation The Bootstrap Weighting and Variance Splines Additive Models Testing Regression Specifications Logistic Regression Generalized Linear Models and Generalized Additive Models Classification and Regression Trees II. Distributions and Latent Structure Density Estimation Relative Distributions and Smooth Tests of Goodness-of-Fit Principal Components Analysis Factor Models Nonlinear Dimensionality Reduction Mixture Models Graphical Models III. Dependent Data Time Series Spatial and Network Data Simulation-Based Inference IV. Causal Inference Graphical Causal Models Identifying Causal Effects Causal Inference from Experiments Estimating Causal Effects Discovering Causal Structure Appendices Data-Analysis Problem Sets Reminders from Linear Algebra Big O and Little o Notation Taylor Expansions Multivariate Distributions Algebra with Expectations and Variances Propagation of Error, and Standard Errors for Derived Quantities Optimization chi-squared and the Likelihood Ratio Test Proof of the Gauss-Markov Theorem Rudimentary Graph Theory Information Theory Hypothesis Testing Writing R Functions Random Variable Generation
Some free Stats textbooks are also available here.
I know other authors have gone to some trouble to make their books available here on stack exchange ... The printed version of our 2002 edition was printed 3 times and sold out 3 times; Springer and Google recently started selling it (book only) as a PDF eBook (no software) on the Springer and Google sites for $79.
We are delighted to be able to make the PDF eBook version (2002 edition) available for FREE to stackexchange users at:
This is a complete PDF version of the original 2002 printed edition. Although no software is included (neither Mathematica nor mathStatica), the methods, theorems, summary tables, examples, exercises, theorems etc are all useful and relevant ... even as a reference text for people who do not even have Mathematica.
One can either download:
the entire book as a single download file ... with live clickable Table of Contents etc, ... or
chapter by chapter.
To install as an iBook:
Download the entire book as a single PDF file
Then drag it into iBooks (under the section: PDF files).
To install on an iPad:
First install it as an iBook (as above)
Open iTunes; select your iPad; click on Books: select the book and sync it over to your iPad.
It's nice to see academics freely distribute their works. Here is trove of free ML / Stats books in PDF:
- Elements of Statistical Learning Hastie, Tibshirani, Friedman
- All of Statistics Larry Wasserman
- Machine Learning and Bayesian Reasoning David Barber
- Gaussian Processes for Machine Learning Rasmussen and Williams
- Information Theory, Inference, and Learning Algorithms David MacKay
- Introduction to Machine Learning Smola and Vishwanathan
- A Probabilistic Theory of Pattern Recognition Devroye, Gyorfi, Lugosi
- Introduction to Information Retrieval Manning, Rhagavan, Shutze
- Forecasting: principles and practice Hyndman, Athanasopoulos (Online Book)
Probability / Stats
- Introduction to statistical thought Lavine
- Basic Probability Theory Robert Ash
- Introduction to probability Grinstead and Snell
- Principle of Uncertainty Kadane
Linear Algebra / Optimization
- Linear Algebra, Theory, and Applications Kuttler
- Linear Algebra Done Wrong Treil
- Applied Numerical Computing Vandenberghe
- Applied Numerical Linear Algebra James Demmel
- Convex Optimization Boyd and Vandenberghe
Statsoft's Electronic Statistics Handbook ('The only Internet Resource about Statistics Recommended by Encyclopedia Britannica') is worth checking out.
Not properly an entire textbook, but the part IV of Mathematics for Computer Science is about probability and random variables.
Some downloadable notes on probability, which seems interesting: http://www.math.harvard.edu/~knill/teaching/math19b_2011/handouts/chapters1-19.pdf
Applied probability: http://www.acsu.buffalo.edu/~bialas/EAS305/docs/EAS305%20NOTES%202005.pdf
A write up of probability tutorials and related puzzles along with R code for learning. Hope it helps
http://www.probabilitycourse.com/ is a website hosting free online-based Probability and Statistics textbook. It also has extra features such as graphing tools and lecture videos
Here is also a great free book on multivariate statistics by Marden, primarily concerned with the normal linear model linked on this page:
Gelman et al. "Bayesian Data Analysis" (3rd edition).
It's not a textbook but Bayesian Methods in the Search for the MH370 is a great introduction to particle filters.
S. Davey, N. Gordon, I. Holland, M. Rutten, and J. Williams, Bayesian Methods in the Search for MH370 (2016)
A digital textbook on probability and statistics by M. Taboga can be found at https://www.statlect.com The level is intermediate. It has hundreds of solved exercises and examples, as well as step-by-step proofs of all the results presented.