2
$\begingroup$

I am struggling to see how to solve the following problem:

I have n i.i.d bernoulli trials. The result can be -1 or 1. I can figure out the expected value of this = n(p-(1-p) but how do I know what the variance is if p is known?

I know that: $$ {\rm Var}(X) = E[X^2] - E[X]^2 $$ I don't know where to go from here.

$\endgroup$
  • 2
    $\begingroup$ In general, for a Bernoulli Random Variable E(X) is $P(X=x_1)*x_1+P(X=x_2)*x_2$. In the "default" case $x_1 = 1, x_2 = 0$, which reduces this to E(X) = p. In your case, $x_1 = 1 $ and $x_2 = -1$. You can then plug in what you get for E(X) into the formula $Var(X) = E(X^2) - E(X)^2$. The variance of the default case is $p*(1-p)$ $\endgroup$ – Pegah Jul 27 '17 at 13:57
  • $\begingroup$ If B is a 0-1 Bernoulli variable with probability of a 1 at $p$, then you're describing a variable, ($Z$ say) where $Z=2B-1$. Just apply basic properties of variance to that linear transformation. $\endgroup$ – Glen_b Jul 28 '17 at 2:06
8
$\begingroup$

When you work with variances, know these facts (in addition to the definition of variance):

  1. The variance of a sum of independent (or just uncorrelated) variables is the sum of their variances.

  2. The expectation of a sum of variables (independent or not) is the sum of their expectations.

  3. The expectation (of any discrete variable) is the sum of each possible value multiplied by its probability.

As an example, let's apply them to your case. You may model the sum of your Bernoulli trials with a variable $X$ expressed as the sum of $n$ independent variables $X_i$. Each $X_i$ takes on the value $1$ with probability $p$ and the value $-1$ with the probability $1-p$.

Fact $(3)$ asserts $$\mathbb{E}(X_i) = p(1) + (1-p)(-1) = 2p-1$$ and $$\mathbb{E}(X_i^2) = p(1)^2 + (1-p)(-1)^2 = p + (1-p) = 1.$$

Fact $(2)$ asserts $$\eqalign{\mathbb{E}(X) &= \mathbb{E}(X_1+\cdots+X_n) = \mathbb{E}(X_1) + \cdots + \mathbb{E}(X_n) = n\mathbb{E}(X_1) \\&=n(2p-1).}$$

The definition of variance now tells you$$\operatorname{Var}(X_i) = \mathbb{E}(X_i^2) - \mathbb{E}(X_i)^2 = 1 - (2p-1)^2 = 4p(1-p).$$

Consequently, fact $(1)$ yields $$\eqalign{\operatorname{Var}(X) &= \operatorname{Var}(X_1+\cdots+X_n) = \operatorname{Var}(X_1) + \cdots + \operatorname{Var}(X_n)=n \operatorname{Var}(X_1) \\&= n(4p(1-p)).}$$


The appearance of the factor of $4$ might seem somewhat mysterious. There's another extremely useful fact you might consider using to shortcut these considerations and identify the origin of that factor:

(4) The variance of a shifted, rescaled variable is the square of the scale factor times the variance. In mathematical symbols, $$\operatorname{Var}(\sigma X + \mu) = \sigma^2 \operatorname{Var}(X)$$ no matter what values the numbers $\sigma$ and $\mu$ might have.

This applies by noting that your $X_i$ can all be expressed as $2Y_i-1$ where $Y_i$ is a true Bernoulli (that is, $0-1$) variable. It's simple to show this: since $2\times 1-1=1$ and $2\times 0 - 1=-1$, the values of $1$ and $0$ taken on by $Y_i$ become $1$ and $-1$, respectively, for $X_i$. The probabilities are unchanged.

You might already know that $Y=Y_1+\cdots + Y_n$, the sum of $n$ independent Bernoulli variables with common probability $p$, is called a Binomial variable. It has a Binomial distribution. You can remember or look up its variance, which is $np(1-p)$. Since $$X = X_1+\cdots+X_n = (2Y_1-1) + \cdots + (2Y_n-1) = 2(Y_1 + \cdots + Y_n) - n=2Y-n,$$ you can take $\sigma=2$ in applying fact $(4)$, which immediately tells you $$\operatorname{Var}(X) = 2^2 \operatorname{Var}(Y) = 4np(1-p).$$

$\endgroup$
  • $\begingroup$ I personally find the observation that $4=\frac{1}{0.5(1-0.5)}$ soothing as it shows that the variance is the ratio of $pq(X_i)$ to $pq(0.5)$ (forgive the abuse of notation). $\endgroup$ – Jared Goguen Jul 27 '17 at 16:09
  • $\begingroup$ In what sense is the second part of the answer $X_i=2Y_i-1$ ... different from my answer ? $\endgroup$ – user83346 Jul 28 '17 at 5:41
  • $\begingroup$ @fcop It's exactly the same approach--but the explanation is more extensive and thorough. If you take a look over the entirety of my answer, I hope you will see the pattern: it focuses on stating and illustrating useful principles. These are often overlooked or unknown to people who come to our site with questions. My purpose is to collect, unify, and illustrate some (obvious, well-known) techniques that can be found by future visitors and referenced in future answers. In that sense, my answer differs from yours, which focuses on solving the immediate problem. $\endgroup$ – whuber Jul 28 '17 at 13:37
  • $\begingroup$ Well you could have referenced it no? $\endgroup$ – user83346 Jul 28 '17 at 14:08
  • $\begingroup$ @fcop One ordinarily does not reference obvious or well-known things. (If that were the practice, then most posts would be 90% references.) The point of this answer is to explain the reasoning--not just the results--at the same level of understanding as the question. Its contribution is not to make the observation that $X_i=2Y_i-1$ (you're welcome to take credit for that), but rather to show as clearly and explicitly as possible, with all reasons supplied, how that produces the same answer derived earlier in the question. $\endgroup$ – whuber Aug 13 '17 at 13:10
1
$\begingroup$

if you take $x=2b-1$ where $b$ is bernoulli, then then $x$ is either 1 or -1, and mean and variance follow from Bernoulli $\mu_b=p$ and $\sigma^2_b=p(1-p)$, the extension to 'Binomial' is then just a sum (assuming independence),

The mean of $x$ is then $\mu_x=2\mu_b-1=2p-1$, variance $\sigma_x^2=4\sigma_b^2=4p(1-p)$,

The sum of $n$ independent such $x$'s has mean $n\cdot(2p-1)$ (your result) and the variance is $4\cdot n \cdot p(1-p)$

EDIT after your question in the comment

It is as @Pegah says: $b$ is the ''usual'' Bernoulli with a success probability $p$ and outcomes 0 and 1. It's expected value is $1 \times p + 0 \times (1-p)=p$ and the variance $p(1-p)$ (also from the definition.

So $b$ is the known Bernoulli with outcome 0 and 1 and $x$ is just a linear function of it. The linear function is such that $x$ has outcomes -1 and 1, and the mean and variance can be obtained from the mean and variance of $b$.

$\endgroup$
  • 2
    $\begingroup$ In fcop's derivation, $\mu_b$ is the expected value of the random variable b, which is, as I understand a "default" binomial random variable (i.e. it can take on values 1 or 0, see also my comment above). Here the expected value is just p. $\endgroup$ – Pegah Jul 27 '17 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.