Understanding the meaning of $n = 1$ in the Bernoulli distribution

Question

I am not very familiar with the Bernoulli distribution and am seeking to understand when to use it rather than other, more common binomial distributions in machine learning models.

I began by reading on Wikipedia, which gives the following definition:

The Bernoulli distribution is a special case of the two-point distribution, for which the two possible outcomes need not be 0 and 1. It is also a special case of the binomial distribution; the Bernoulli distribution is a binomial distribution where n=1.

It then proceeds to get more technical, but I don't see it clearly referring back to that statement later in the article. I'm just looking for a basic, high-level understanding but what confuses me about the definition above is that it says "where n = 1".

Which "n" is this referring to? It was not defined in the Wikipedia summary, but lacking another definition I usually take n to refer to sample size. Is that saying the Bernoulli distribution is for when you have a sample size of 1? That doesn't make any sense to me and seems to conflict with some empirical examples I've seen of the Bernoulli distribution being using in Generalized Boosting Models, thus I'm confused.

Answer 1

Bernoulli trials are experiments with binary outcomes: success or failure. One way to represent that is as 0 and 1. Or we could think about a coin toss: it's a head (H) or tails (T). The H/T representation is not numerical, but contains precisely the same amount of information about the experiment as the 0 and 1 encoding.

The choice of 0 and 1 is convenient because the mean of a set of 0 and 1-encoded data is the proportion of 1's. But one could conceivably use -1, 1 encoding, or 1, 2-encoding; in a somewhat obscure set of circumstances, these alternative encoding schemes might even be helpful.

The $n$ parameter is referring to the number of trials in a binomial distribution. The binomial distribution counts the number of experimental outcomes of one binary type (conventionally, it counts the successes). So if a sequence of coin tosses is recorded as HTTT, then we might say that there were 3 successes (if we define heads as successes; this is also a convention). We can parameterize the binomial distribution as a number of trials and a probability of success (when the trials are statistically independent, the probability is fixed at the same value for all trials, and the number of trials is finite). Clearly, if $n$ is 1, we have a Bernoulli distribution.

See also: Why 0 for failure and 1 for success in a Bernoulli distribution?

Answer 2

The Binomial distribution models the number of "successes" $k$ in $n$ trials. Success can, of course, be defined in any suitable way, like the number of times your favorite animal (like Paul, the Octpus, pictured below) is going to correctly forecast the outcomes of $n$ different knock-out games (because there can be no ties) at the European championship (to take a current event as of the time of writing) given that it has a probability of $p$ of correctly forecasting any given game. (Many, but not all, people expect $p=0.5$ to be natural here...)

The Bernoulli then corresponds to the case in which just one trial has been made, like in the example of one coin being tossed - it either comes up head or not.

You are of course right that many samples can be taken from the Bernoulli (like from any other) distribution. For example, you may toss the coin many times.

Answer 3

Imagine you have a thumbtack with a $p$ probability of landing on its back when you flip it. (There are two possibilities: thumbtack lands on its back or on its side.)

• Let $X$ denote the outcome of this flip of a thumbtack.
• Let $X=1$ denote the outcome where the thumbtack lands on its back.
• Let $X=0$ denote the outcome where the thumbtack lands on its side.

$X$ would a random variable following the Bernoulli distribution. The Bernoulli distribution is used to represent a binary outcome with $p \in [0, 1]$ probability of success.

Now let's imagine we flip the same thumbtack $n$ times, under the same conditions, with each flip denoted by $X_1$, $X_2$ etc.... Let $Y = \sum_{i=1}^n X_i$ be the number of successes, the number of times the thumbtack landed on its back. (We want the same conditions, same thumbtack, conditions such that $X_1$, $X_2$ etc... are independently and identically distributed with the same probability of success: the scalar value $p$.)

$Y$ would then follow the binomial distribution. Also observe that for $n=1$, for a single flip of a tack, $X_1$ (the result of a single flip) and $Y$ (the number of successful flips) are exactly the same thing.