7
$\begingroup$

Let $S = \{s_{1}, ..., s_{n}\}$ be a finite set. Let $\mathcal{B}$ be any sigma algebra of subsets of $S$. Let $p_{1}, ..., p_{n}$ be nonnegative numbers that sum to 1. For any $A \in \mathcal{B}$, $P(A) = \sum_{\{s_{i} \in A\}} p_{i})$, where $s_{i}$ are sample points.

Which says the probability of any given set is equal to the sum of the probability of its sample points

Can this rule ever be broken?

$\endgroup$
2
  • 1
    $\begingroup$ It's unclear what you might mean by "this rule." One interpretation asks whether there exist $p_i$ for which $P(A)$ is given by such a formula. Linear algebra tells us yes. It also tells us the $p_i$ are unique iff $\mathcal B$ is the power set of $S.$ I believe this result extends to all discrete probability measures, but that requires a little more analysis to show. $\endgroup$
    – whuber
    Commented Jan 15 at 18:54
  • $\begingroup$ When you are saying “where $s_{i}$ are sample points.”, you are effectively saying that they are mutually exclusive events and that Kolmogorov's third axiom, $\sigma$-aditivity, applies to it. $\endgroup$ Commented Jan 16 at 9:25

3 Answers 3

11
$\begingroup$

By $p_i, $ I presume $\Pr(\{s_i\}) :=p_i.$ This means the singletons $\{s_i\}\in \mathcal B$ and as $\sum p_i=1, $ this would also mean $\bigsqcup \{s_i\} = S, $ (which is obvious).

So, the singletons are atoms which, in general, means a measurable set $A$ such that only subsets of $A$ which are measurable are $\emptyset$ and $A$ itself. If $A_i$ are disjoint and $\bigsqcup A_i=S, $ then $\sigma(A_1, A_2, \ldots, A_n) $ is constructed by taking unions of the atoms; in-fact the cardinality would be $2^n.$

So, any non-empty $A\in\mathcal B$ would be of the form $A=\sqcup_{j=1}^k \{s_j\}; 1\leq k\leq n. $ Hence $\Pr(A) =\sum_j p_j.$

$\endgroup$
2
  • 2
    $\begingroup$ +1. The question does not assume that $P(\{s_j\})=p_j$ (though it is probably what the OP had in mind). There exist $\sigma$-algebras where some singletons are not measurable (hence have no probability). For example, $\mathcal B=\{ \emptyset, \{1,2\}, \{3,4\}, \{1,2,3,4\} \}$. $\endgroup$
    – Taladris
    Commented Jan 15 at 12:11
  • $\begingroup$ When writing the comment, I had only the first paragraph in mind. Actually, the assumption $P(\{s_j\})=p_j$ can be inferred from the title and the second paragraph. $\endgroup$
    – Taladris
    Commented Jan 15 at 23:12
11
$\begingroup$

An atomic event is any nonempty event $\mathcal E\in \mathcal B$ that is as small as possible: that is, if $\mathcal F\in \mathcal B$ and $\mathcal F\subset \mathcal E$ with $\mathcal F\ne \mathcal E,$ then $\mathcal F = \emptyset.$

A finite (sigma)-algebra $\mathcal B$ has only a finite number of events. (The modifier "sigma" is thereby superfluous.)

Atomic events always exist in finite algebras. (You can prove this with the method of descent. It the assertion were false, there would be a finite algebra with no atomic events. Such an algebra obviously is nontrivial (it has more than two events). Pick any proper nonempty event $\mathcal A$ in this algebra. It induces a subalgebra whose events are the intersections of the original events with $\mathcal A.$ By hypothesis it must have an atomic event, which (obviously) is also an atomic event in the original algebra, contradicting the assumption.)

Because intersections of events are also events, the intersection of two atomic events cannot be a proper subset of either event. Consequently, each outcome $\omega$ is contained in a unique atomic event. Let's call that event $\mathcal A(\omega).$

There exists at least one function $P:\mathcal A(\omega)\to [0,1]$ (the real numbers between $0$ and $1$) for which

$$\sum_{\eta\in\mathcal A(\omega)} p(\eta) = \Pr(A(\omega)).$$

For example, the function that assigns the entire probability $\Pr(\mathcal A(\omega))$ to $\omega$ and otherwise assigns $0$ to all other elements of $\mathcal A(\omega)$ will work.

Upon performing this construction for all the (finitely many) atomic events, we will have defined a function $p:\Omega\to [0,1]$ assigning nonzero numbers to every outcome in the sample space $\Omega.$ The sum of all these numbers is the sum of the probabilities of the atomic sets, which (by one of the axioms of probability) equals $1.$

This construction has demonstrated the following:

In any finite algebra $\mathcal B$ defined on a sample space $\Omega,$ there exists a function $p:\Omega\to[0,1]$ for which whenever $\mathcal E\in\mathcal B$ is an atomic event, then $$\Pr(\mathcal E) = \sum_{\omega\in\mathcal E}p(\omega).$$

Since every event is a union (uniquely) of atomic events, the same conclusion holds for all events. That's similar to the statement in the question. However, we cannot use the term "probability" for the values of $p.$ The reason is that the singletons $\{\omega\}$ are not necessarily events. (See the example below, where the only singleton that is an event is $\{0\}.$) Only the original events $\mathcal B$ can be said to have well-defined probabilities.

In English, then,

When the sigma algebra $\mathcal B$ is finite, there exists at least one function $p$ for which the probability of any event is equal to the sum of the values of $p$ at its sample points.


As a concrete example of how this works, let $\Omega=\mathbb R$ be the set of real numbers and let the atomic events be the negative numbers $\mathbb R_-,$ the positive numbers $\mathbb R_+,$ and $\{0\}.$ Assign probabilities to these three events. (They must be nonnegative values summing to $1.$ For example, pick $1/3,$ $1/3,$ and $1/3.$)

The sigma algebra $\mathcal B$ has eight elements. They can be characterized as

  • The empty set $\emptyset.$
  • The negative numbers $\mathbb R_-.$
  • The non-positive numbers $\mathbb R_-\cup \{0\}.$
  • The positive numbers $\mathbb R_+.$
  • The non-negative numbers $\mathbb R_+\cup \{0\}.$
  • Zero, $\{0\}.$
  • The non-zero numbers $\mathbb R_- \cup \mathbb R_+.$
  • All real numbers $\mathbb R.$

To define the $p_i,$ pick your favorite negative number, your favorite positive number, and $0:$ these play the role of $\omega$ in the construction. For instance, I selected $-1,$ $1,$ and $0.$ Set $p_{-1} = \Pr(\mathbb R_-),$ $p_{1} = \Pr(\mathbb R_+),$ and $p_0 = \Pr(\{0\}).$ For any real number $x$ not equal to $-1,$ $0,$ or $1,$ set $p_x=0.$ You can verify now that the probabilities of each of the eight events listed above is the sum of $p_x$ as $x$ ranges over the event (because these are all sums with only finitely many nonzero values).

Finally, because your favorite numbers might not be my favorite numbers, the two of us might arrive at different values of $p.$ Both solutions work: they give the same, original probabilities for all events in $\mathcal B.$ But, because the details of the function $p$ are subjective, its values cannot be termed "probabilities" in this context.

$\endgroup$
8
  • 2
    $\begingroup$ Great answer! Our answers are similar, but my memory of probability theory is fading, so I learned/reviewed a lot through yours. :) $\endgroup$
    – Taladris
    Commented Jan 15 at 23:14
  • $\begingroup$ I do not understand the point of the example where the $p_x$ seem to be infinite in number whereas the question is about a finite set of sample points. $\endgroup$ Commented Jan 16 at 9:22
  • $\begingroup$ @SextusEmpiricus The implicit point is that the cardinality of the sample space is irrelevant: this is a discussion about finite algebras. There is some potential confusion about that (lurking in hidden assumptions in some other posts in this thread), which is why I specifically chose an uncountably infinite sample space. But that's not the main point, so I did not make an issue of it. $\endgroup$
    – whuber
    Commented Jan 16 at 14:59
  • $\begingroup$ @whuber I didn't really get the question either. Is your answer some specific interpretation/extension of it? How is the question a discussion about finite algebras? $\endgroup$ Commented Jan 16 at 16:19
  • $\begingroup$ The statement in the question that seems to be the point of the discussion is “For any $A \in \mathcal{B}$, $P(A) = \sum_{\{s_{i} \in A\}} p_{i}$”. If that is right, then to me this looks like a trivial case since, if we assume this question to be about a probability space $(S,\mathcal{B},P)$, then the statement is just Kolmogorov's third axiom, and the statement is true by assumption. $\endgroup$ Commented Jan 16 at 16:25
7
$\begingroup$

Note: this is a complement to @User1865345's answer.

It is not clear whether the OP assumes $p_j=P(\{s_j\})$ for every $j\in \{1,2,\dots,n\}$ or not. It is not clear either that $\{s_j\}$ is assumed to be measure ($\mathcal B$ is any $\sigma$-algebra on $S$). For example, $\mathcal B=\{\emptyset,\{1,2\},\{3,4\},\{1,2,3,4\}\}$ is a $\sigma$-algebra on $\{1,2,3,4\}$.

If $\{s_j\}$ is measurable and $p_j=P(\{s_j\})$ for every $j\in \{1,2,\dots,n\}$, then @User1865345 gave a perfect answer.

Without this assumption, the question can be understood as

Let $S$ be a finite set with $n$ elements and $\mathcal B$ a $\sigma$-algebra on $S$, does it exist a finite sequence $p_1$, $\dots$, $p_n$ of nonnegative numbers such that $$\tag{1} P(A) = \sum_{s_j\in S}p_j $$ for every $A$ in $\mathcal B$?

The answer is yes. If $s$ is an element of $S$, we can define $M(s)$ as the intersection of all elements of $\mathcal B$ that contains $s$. It is well-known that

  1. $M(s)$ is measurable for every $s$ in $S$
  2. $F=\{M(s)| s\in S \}$ forms a partition of $S$.
  3. Every measurable set of $\mathcal B$ is a disjoint union of elements of $F$ (including the empty set as the empty union).

In the example above, $F=\{ \{1,2\}, \{3,4\} \}$. Note that $F$ is a set, so repetitions have been removed (in the example, $M(1)=M(2)=\{1,2\}$).

Now, if $j\in\{1,2,\dots,n\}$, define

$$p_j=\frac{P(M(s_j)}{|M(s_j)|}$$

(where $|M(s_j)|$ is the number of elements of the set $X$). The three properties above and computations similar to @User1865345 imply that $P(A)=\sum_{s_j\in S}{p_j}$ for every $A$ in $\mathcal B$.

In the construction above, we used the axioms on which probability theory relies. In particular, we used the following axiom to obtain the formula $(1)$:

Let $S$ be a finite set and $\mathcal B$ a $\sigma$-algebra on $S$. If $A_1$, $A_2$, $\dots$, $A_k$ are a finite number of pairwise disjoint elements of $\mathcal B$, then $$P(A_1\cup A_2\cup \dots \cup A_k)=P(A_1)+P(A_2)+\dots+P(A_k)$$

(If the sample space is infinite, finite is replaced by at most countable).

Like any theory, the axioms or principles can be discussed. However, the axiom above is quite intuitive. When $k=2$, it means that the overall probability, that is the measure of likelihood, of two events that cannot happen at the same time, is the sum of their individual probabilities.

This is a very strong axiom of probability theory (and more generally of measure theory), and I think that what we obtain when removing it cannot be called probability anymore.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.