647

The short version is that the Beta distribution can be understood as representing a distribution of probabilities- that is, it represents all the possible values of a probability when we don't know what that probability is. Here is my favorite intuitive explanation of this: Anyone who follows baseball is familiar with batting averages- simply the number of ...


238

Let me first explain what a conjugate prior is. I will then explain the Bayesian analyses using your specific example. Bayesian statistics involve the following steps: Define the prior distribution that incorporates your subjective beliefs about a parameter (in your example the parameter of interest is the proportion of left-handers). The prior can be "...


176

First, here are some quick comments: The $p$-values of a Kolmovorov-Smirnov-Test (KS-Test) with estimated parameters will be quite wrong. So unfortunately, you can't just fit a distribution and then use the estimated parameters in a Kolmogorov-Smirnov-Test to test your sample. Your sample will never follow a specific distribution exactly. So even if your $p$...


164

The difficulty with using histograms to infer shape While histograms are often handy and sometimes useful, they can be misleading. Their appearance can alter quite a lot with changes in the locations of the bin boundaries. This problem has long been known*, though perhaps not as widely as it should be -- you rarely see it mentioned in elementary-level ...


114

Log-scale informs on relative changes (multiplicative), while linear-scale informs on absolute changes (additive). When do you use each? When you care about relative changes, use the log-scale; when you care about absolute changes, use linear-scale. This is true for distributions, but also for any quantity or changes in quantities. Note, I use the word "...


101

You can mechanically check that the expected value does not exist, but this should be physically intuitive, at least if you accept Huygens' principle and the Law of Large Numbers. The conclusion of the Law of Large Numbers fails for a Cauchy distribution, so it can't have a mean. If you average $n$ independent Cauchy random variables, the result does not ...


69

The Dirichlet distribution is a multivariate probability distribution that describes $k\ge2$ variables $X_1,\dots,X_k$, such that each $x_i \in (0,1)$ and $\sum_{i=1}^N x_i = 1$, that is parametrized by a vector of positive-valued parameters $\boldsymbol{\alpha} = (\alpha_1,\dots,\alpha_k)$. The parameters do not have to be integers, they only need to be ...


62

Mean = mode doesn't imply symmetry. Even if mean = median = mode you still don't necessarily have symmetry. And in anticipation of the potential followup -- even if mean=median=mode and the third central moment is zero (so moment-skewness is 0), you still don't necessarily have symmetry. ... but there was a followup to that one. NickT asked in comments ...


60

Yes, there are many ways to produce a sequence of numbers that are more evenly distributed than random uniforms. In fact, there is a whole field dedicated to this question; it is the backbone of quasi-Monte Carlo (QMC). Below is a brief tour of the absolute basics. Measuring uniformity There are many ways to do this, but the most common way has a strong, ...


54

The mean and variance are defined in terms of integrals. What it means for the mean or variance to be infinite is a statement about the limiting behavior for those integrals For example, the mean is $\lim_{a,b\to\infty}\int_{-a}^b x\ dF$ (considering this, say as a Stieltjes integral); for a continuous density this would be $\lim_{a,b\to\infty}\int_{-a}^b ...


53

You may be looking for distribution known under the names of generalized normal (version 1), Subbotin distribution, or exponential power distribution. It is parametrized by location $\mu$, scale $\sigma$ and shape $\beta$ with pdf $$ \frac{\beta}{2\sigma\Gamma(1/\beta)} \exp\left[-\left(\frac{|x-\mu|}{\sigma}\right)^{\beta}\right] $$ as you can notice, for ...


51

Adding to the other excellent answers, an answer with another viewpoint which maybe can add some more intuition, which was asked for. The Kullback-Leibler divergence is $$ \DeclareMathOperator{\KL}{KL} \KL(P || Q) = \int_{-\infty}^\infty p(x) \log \frac{p(x)}{q(x)} \; dx $$ If you have two hypothesis regarding which distribution is generating the data $X$,...


51

If you "know" that the coin is fair then we still expect the long run proportion of heads to tend to $0.5$. This is not to say that we should expect more (than 50%) of the next flips to be tails, but rather that the initial $500$ flips become irrelevant as $n\rightarrow\infty$. A streak of $500$ heads may seem like a lot (and practically speaking it is), ...


49

A Beta distribution is used to model things that have a limited range, like 0 to 1. Examples are the probability of success in an experiment having only two outcomes, like success and failure. If you do a limited number of experiments, and some are successful, you can represent what that tells you by a beta distribution. Another example is order statistics....


49

The saddlepoint approximation to a probability density function (it works likewise for mass functions, but I will only talk here in terms of densities) is a surprisingly well working approximation, that can be seen as a refinement on the central limit theorem. So, it will only work in settings where there is a central limit theorem, but it needs stronger ...


45

The Beta distribution also appears as an order statistic for a random sample of independent uniform distributions on $(0,1)$. Precisely, let $U_1$, $\ldots$, $U_n$ be $n$ independent random variables, each having the uniform distribution on $(0,1)$. Denote by $U_{(1)}$, $\ldots$, $U_{(n)}$ the order statistics of the random sample $(U_1, \ldots, U_n)$, ...


44

The (right) tail of a distribution describes its behavior at large values. The correct object to study is not its density--which in many practical cases does not exist--but rather its distribution function $F$. More specifically, because $F$ must rise asymptotically to $1$ for large arguments $x$ (by the Law of Total Probability), we are interested in how ...


43

A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.


41

Answer added in response to @whuber's comment on Michael Chernicks's answer (and re-written completely to remove the error pointed out by whuber.) The value of the integral for the expected value of a Cauchy random variable is said to be undefined because the value can be "made" to be anything one likes. The integral $$\int_{-\infty}^{\infty} \frac{x}{\...


41

You have stumbled upon one of the most famous results of probability theory and statistics. I'll write an answer, although I am certain this question has been asked (and answered) before on this site. First, note that the pdf of $Y = X^2$ cannot be the same as that of $X$ as $Y$ will be nonnegative. To derive the distribution of $Y$ we can use three methods,...


41

Exponential distributions are often used to model survival times because they are the simplest distributions that can be used to characterize survival / reliability data. This is because they are memoryless, and thus the hazard function is constant w/r/t time, which makes analysis very simple. This kind of assumption may be valid, for example, for some kinds ...


41

I agree with X'ian that the problem is under-specified. However, there is an elegant, scalable, efficient, effective, and versatile solution worth considering. Because the product of the sample mean and sample size equals the sample sum, the problem concerns generating a random sample of $n$ values in the set $\{1,2,\ldots, k\}$ that sum to $s$ (assuming $...


40

Because, assuming normal errors is effectively the same as assuming that large errors do not occur! The normal distribution has so light tails, that errors outside $\pm 3$ standard deviations have very low probability, errors outside of $\pm 6$ standard deviations are effectively impossible. In practice, that assumption is seldom true. When analyzing ...


39

I consider the following linear model: ${y} = X \beta + \epsilon$. The vector of residuals is estimated by $$\hat{\epsilon} = y - X \hat{\beta} = (I - X (X'X)^{-1} X') y = Q y = Q (X \beta + \epsilon) = Q \epsilon$$ where $Q = I - X (X'X)^{-1} X'$. Observe that $\textrm{tr}(Q) = n - p$ (the ...


39

First, combine any sums having the same scale factor: a $\Gamma(n, \beta)$ plus a $\Gamma(m,\beta)$ variate form a $\Gamma(n+m,\beta)$ variate. Next, observe that the characteristic function (cf) of $\Gamma(n, \beta)$ is $(1-i \beta t)^{-n}$, whence the cf of a sum of these distributions is the product $$\prod_{j} \frac{1}{(1-i \beta_j t)^{n_j}}.$$ When ...


39

All this may sound complicated at first, but it is essentially about something very simple. By cumulative distribution function we denote the function that returns probabilities of $X$ being smaller than or equal to some value $x$, $$ \Pr(X \le x) = F(x).$$ This function takes as input $x$ and returns values from the $[0, 1]$ interval (probabilities)&...


39

No. Take $X$ uniform on $[-1,1]$ and $a=0$. Then $\Pr(X < a ) = 1/2$. On the other hand $\Pr(X^2<a^2) = 0$.


39

While a number of posts on site address various properties of the Cauchy, I didn't manage to locate one that really laid them out together. Hopefully this might be a good place to collect some. I may expand this. Heavy tails While the Cauchy is symmetric and roughly bell shaped, somewhat like the normal distribution, it has much heavier tails (and less of ...


38

Let $X\sim N(0,1)$ and define $Y=-X$. It is easy to prove that $Y\sim N(0,1)$. But $$ P\{\omega : X(\omega)=Y(\omega)\} = P\{\omega : X(\omega)=0,Y(\omega)=0\} \leq P\{\omega : X(\omega)=0\} = 0 \, . $$ Hence, $X$ and $Y$ are different with probability one.


Only top voted, non community-wiki answers of a minimum length are eligible