# Why is the likelihood of a Bernoulli variable modelled by a product of the mean parameter?

I've seen that the likelihood function of the parameter $\lambda$ of a Bernoulli variable given the training data $\boldsymbol X$ with labels $\boldsymbol y$ is modeled like this:

$$p(\boldsymbol y\mid\boldsymbol X;\lambda)=\prod\limits_{i=1}^I\lambda^{y_i}(1-\lambda)^{1-y_i}$$

I understand that if we consider the negative logarithm of this as a loss of an optimization problem, then the parameter is correctly adapted because if the ground truth label is $y_i=1$ then the exponent selects the first term and in order to maximize the likelihood it needs to be maximized (until it matches the ground truth label 1), and if $y_i=0$ the exponent selects $(1-\lambda)$, so in order to maximize the likelihood, $\lambda$ needs to be minimized (until it matches the ground truth label 0). Can this likelihood function be derived from first principles, e.g. using the fact that $\lambda$ is the mean of the distribution?

In essence you take the product because of the product rule for the probability of the intersection of independent events (i.e. $P(AB) = P(A) \, P(B)$, but here taken over more than two events).

See the corresponding rule for random variables; in the case of the Bernoulli we'd be using $f$ to represent the pmf rather than a density. That is just a consequence of the basic probability rule I mentioned in the first paragraph.

Then considering the definition of likelihood, the result follows immediately.

Suppressing the $X$ for clarity and using your $p$ rather than Wikipedia's $f$ to better emphasize the discreteness:

\begin{align} L(\lambda) & = p(\mathbf{y}\mid\lambda) & & _\text{(definition)} \\[10pt] & = p(y_1\mid\lambda)\times p(y_2\mid\lambda)\times \cdots \times p(y_n\mid\lambda) & & _\text{(product rule)} \end{align}

This is not just true for the Bernoulli of course; it's used for likelihood any time you have independence.

• @LenarHoyt why what..? Why this is the formula for Bernoulli probability mass function?
– Tim
Feb 23 '17 at 9:06