I know very little of Probability and Statistics, and am wishing to learn. I see the word "distribution" used all over the place in different contexts.

For example, a discrete random variable has a "probability distribution." I know what this is. A continuous random variable has a probability density function, then for $x\in\mathbb{R}$, the integral from $-\infty$ to $x$ of the probability density function is the cumulative distribution function evaluated at $x$.

And apparently just "distribution function" is synonymous with "cumulative distribution function," at least when talking about continuous random variables (question: are they always synonyms?).

Then there are many famous distributions. $\Gamma$ distribution $\chi^2$ distribution, etc. But what exactly is a $\Gamma$ distribution? Is it the cumulative distribution function of a $\Gamma$ random variable? Or the probability density function of a $\Gamma$ random variable?

But then a frequency distribution of a finite data set appears to be a histogram.

Long story short: in Probability and Statistics, what is the definition of the word "distribution"?

I know the definition of distribution in Mathematics (an element of the dual space of the collection of test functions equipped with the inductive limit topology), but not Probability and Statistics.

  • 1
    $\begingroup$ The corresponding Wikipedia article seems to be a decent introduction into the topic. $\endgroup$ Apr 18, 2015 at 1:01
  • 1
    $\begingroup$ Strictly, 'distribution' and 'cdf' should be regarded as synonyms, but 'distribution' is often used in a much looser sense and often is used to actually refer to a density/pmf. $\endgroup$
    – Glen_b
    Apr 18, 2015 at 6:01
  • 3
    $\begingroup$ Your understanding of a distribution is pretty close to the one in probability; the main difference is that those in probability enjoy some additional properties (of being positive and normalized to unity). The connection is that your definition establishes a distribution in terms of the associated expectation operator. There is also a (serious) abuse of language prevalent in statistics, which also calls a parameterized family of distributions a "distribution." Finally, any finite dataset determines a distribution obtained by sampling from it, its "empirical distribution." $\endgroup$
    – whuber
    Apr 19, 2015 at 0:01
  • $\begingroup$ @whuber That helps, thanks In particular, the abuse of language. It'd be like calling the indefinite integral of a function... a function. $\endgroup$
    – danzibr
    Apr 19, 2015 at 22:12
  • $\begingroup$ A similar question with good answers: stats.stackexchange.com/questions/210403/… $\endgroup$ Jun 12, 2018 at 6:44

3 Answers 3


The following is for $\mathbb R-$valued random-variables. The extension to other spaces is straight forward if you are interested. I would argue that the following slightly more general definition is more intuitive than separately considering density, mass and cumulative distribution functions.

I include some mathematical / probabilistic terms in the text to make it correct. If one is not familiar with those terms, the intuition is equally well grasped by just thinking of "Borel sets" as "any subset of $\mathbb R$ that I can think of", and of the random variable a the numerical outcome of some experiment with an associated probability.

Let $\left( \Omega, \mathcal F, P \right)$ be a probability space and $X(\omega)$ an $\mathbb R-$valued random variable on this space.

The set function $Q(A):=P\left(\omega \in \Omega : X(\omega) \in A\right)$, where $A$ is a Borel set, is called the distribution of $X$.

In words, the distribution tells you (loosely speaking), for any subset of $\mathbb R $, the probability that $X$ takes on a value in that set. One can prove that $Q$ is completely determined by the function $F(x):=P(X\leq x)$ and vice versa. To do that -- and I skip the details here -- construct a measure on the Borel sets that assign the probability $F(x)$ to all sets $(-\infty, x)$ and argue that this finite measure agrees with $Q$ on a $\pi-$system generating the Borel $\sigma-$algebra.

If it so happens that $Q(A)$ can be written as $Q(A) =\int_Af(x)dx$ then $f$ is a density function for $Q$ and you can see, although this density is not uniquely determined (consider changes on sets of Lebesgue measure zero), it makes sense to also speak of $f$ as the distribution of $X$. Usually, however, we call it the probability density function of $X$.

Similarly, if it so happens that $Q(A)$ can be written as $Q(A)=\sum_{i\in A\cap\{\dots,-1,0,1,\dots\}}f(i)$, then it makes sense to speak of $f$ as the distribution of $X$ although we usually call it the probability mass function.

Thus, whenever you read something like "$X$ follows a uniform distribution on $[0,1]$", it simply means that the function $Q(A)$, which tells you the probability that $X$ takes on values in certain sets, is characterized by the probability density function $f(x)=I_{[0,1]}$ or the cumulative distribution function $F(x)=\int_{-\infty}^x f(t)dt$.

A final note on the case where there is no mention of a random variable, but only a distribution. One may prove that given a distribution function (or a mass, density or cumulative distribution function), there exists a probability space with a random variable that has this distribution. Thus, there is essentially no difference in speaking about a distribution, or about a random variable having that distribution. It's just a matter of one's focus.


Let $(\Omega,\mathscr{F},P)$ be a probability space, let $(\mathscr{X},\mathscr{B})$ be a measurable space, and let $X:\Omega\to\mathscr{X}$ be a measurable function, which means that $X^{-1}(B)=\{\omega:X(\omega)\in B\}\in\mathscr{F}$ for every $B\in\mathscr{B}$. The distribution of $X$ is the probability measure $\mu_X$ over $(\mathscr{X},\mathscr{B})$ defined by $\mu_X(B)=P(X\in B)$. When $\mathscr{X}=\mathbb{R}$ and $\mathscr{B}$ is the Borel sigma-field, we refer to the function $X$ as a random "variable".

  • 1
    $\begingroup$ must be very clear to people with little knowledge of probability and statistics :) $\endgroup$ Apr 22, 2015 at 6:55
  • 3
    $\begingroup$ Well, the OP seems to know advanced math stuff such as "element of the dual space of the collection of test functions equipped with the inductive limit topology". Check the end of his question. $\endgroup$
    – Zen
    Apr 22, 2015 at 15:22
  • 2
    $\begingroup$ It was indeed a good response for me. I needed to check the definition of a probability space, but for a person with a math background, it was clear. I appreciated the answer's concision, only not accepting it due to the detail in the other answer. $\endgroup$
    – danzibr
    Apr 28, 2015 at 18:29

The question and answers so far seem to have focused on theoretical distributions. Empirical distributions provide a more intuitive understanding of distributions.


During a class tournament in skipping rope we observe all the kids in a class skipping rope. The first kid is able to jump twice, the second four times, the next one fifteen times, etc. We record the number of jumps. Five of the kids jumped eight times each, but only one of the kids jumped twice. We say that jumping eight times is differently distributed than jumping twice.

An ostensive definition for an observed distribution is the frequency of occurrences for each observed value of a variable.

In inferential statistics we then try to fit theoretical distributions to the observed distributions, because we would like to work with the assumptions of the theoretical distributions. You can reach a similar definition for theoretical distributions by replacing "observed" with "observerable" or to be more precise: "expected".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.