18
$\begingroup$

I'm studying machine learning and every book I open I bump into chi-squared distribution, gamma-function, t-distribution, Gaussian, etc.

Every book I have opened so far only defines what the distributions are: they don't explain or give the intuition on where the specific formulas for the functions come from.

For example, why is chi-squared distribution the way it is? What is the t-distribution? What is the intuition behind the distribution? Proofs? etc.

I would like to have a clear and fundamental understanding of the most commonly used distributions so that every time later on when I see them, I truly understand what is a t-distribution, what is a Gaussian distribution and most importantly why are they the way they are.

It would be nice if the books / tutorials can explain the concepts to a layman so that in order to understand them you don't already need to understand them x) Many books are like this, they don't fit for beginners :(

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Most undergraduate texts on theoretical statistics or probability theory have a chapter on distribution theory that covers these questions. But how much mathematical background would you want to assume? $\endgroup$
    – Scortchi ♦
    Commented Jul 2, 2013 at 10:05
  • $\begingroup$ undergraduate mathematical background :) The fundamental building blocks. Is that sufficient? What kind of level of mathematics should I acquire before learning about the distributions? I have read a basic book about statistics, which only shortly presented the distributions I described in the question. $\endgroup$
    – jjepsuomi
    Commented Jul 2, 2013 at 10:07
  • $\begingroup$ Some probability theory & calculus ought to do it - it depends how deep you want to go. $\endgroup$
    – Scortchi ♦
    Commented Jul 2, 2013 at 11:35
  • $\begingroup$ Okay, thank you :) Mostly I'd just want to understand what I'm doing $\endgroup$
    – jjepsuomi
    Commented Jul 2, 2013 at 11:48
  • 1
    $\begingroup$ You also might find useful the references posted in this thread: stats.stackexchange.com/questions/56385/…. $\endgroup$
    – Andre Silva
    Commented Jul 2, 2013 at 13:02

3 Answers 3

Reset to default
11
$\begingroup$

If you've no mathematical impediments there's a good overview in Ch. 3 of Casella & Berger, Statistical Inference, & much is covered in Grinstead & Snell, Introduction to Probability (it's free); for more detail I'd recommend Severini, Elements of Distribution Theory. But there are lots - it would be more difficult, I think, to find a less mathematical treatment that still gives the reader some feel for where different distributions come from.

Improve this answer
$\endgroup$
6
  • $\begingroup$ why it doesn't have "probability mass function" ? $\endgroup$
    – Sadegh
    Commented Oct 3, 2016 at 21:36
  • 1
    $\begingroup$ Perhaps the OP is looking for a book like this amazon.com/…... the coverage seems to be great but I cannot speak to its quality ... or this one amazon.com/… $\endgroup$
    – ColorStatistics
    Commented Mar 22, 2020 at 23:15
  • $\begingroup$ It is puzzling why this answer was marked as answer of choice when it doesn't answer the question. Casella & Berger has only 50 pages on it and doesn't cover many of the common distributions. Grinstead & Snell spend only 40 pages on it, the usual treatment and emphasis in most statistics books out there. Finally, Severini's book doesn't catalogue and explain the distributions any more than any other mathematical statistics book.... Larsen and Marx's "An Introduction to Mathematical Statistics and Its Appliations", also dedicates 50 pages and its less mathematical than Cassella & Berger $\endgroup$
    – ColorStatistics
    Commented Mar 22, 2020 at 23:31
  • $\begingroup$ @ColorStatistics: I suppose I was taking the question to ask as much for an introduction to distribution theory as for a compendium of brand-name distribution families. Your suggestions certainly seem worth putting into an answer. For book recommendations in particular, there often doesn't seem much good reason for OPs to accept one answer rather than another; but note that they can change their mind to accept a more useful one that appears later. $\endgroup$
    – Scortchi ♦
    Commented Mar 24, 2020 at 10:14
  • $\begingroup$ @Scortchi: I've actually come around and think your answer is on point. There are other books that cover the topic more comprehensively but, I've come to conclude, not more intuitively than the sources you've mentioned. I just up-voted your answer myself; and your point that the books you mention cover distribution theory as well as give a compendium of distribution families, is another good one. $\endgroup$
    – ColorStatistics
    Commented Mar 24, 2020 at 15:46
5
$\begingroup$

You should read "Continuous univariate distributions" Vol. 1 & 2. by Johnson and Kotz. Also "The Weibull distribution A Handbook" by Horst Rinne. Second one is a useful book to understand a distribution although this book focus on Weibull distribution. May be some material is not easy to under stand but early chapters give you some useful knowledge.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The reviews for the recommended books are poor. People complain of many typos. $\endgroup$
    – ColorStatistics
    Commented Mar 22, 2020 at 23:12
2
$\begingroup$

For a short and easy overview of a lot of probability distributions, I recommend Probability and statistics EBook. Most distributions are described in chapter XV, but the more common ones are spread out in earlier parts of the book.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.