Imagine I'm running a series of independent trials where the outcome $X_{i}$ can be either 1 or 0 (a bernoulli variable) with probability $p$.

I will run the trials as in a negative binomial way: I will run as many trials as I need until I get, say, $3$ success.

Question: what is the marginal probability of $X_4$ (the result of the fourth trial)? By marginal, I mean not conditioning on $X_1, X_2, X_3$.

  • $\begingroup$ You'd stated the trials are independent, so if a 4th trial does occur, it's marginal probability is still $p$. Are you looking for the probability this 4th trial occurs, which is $P(n >= 4)$ using the "n trials given k successes" formulation of the negative binomial? $\endgroup$ Nov 23, 2018 at 7:46
  • $\begingroup$ @deasmhumnha I know how to compute the numerical probabilities, the problem with your statement is that a trial only occurs conditional on $X_1, X_2, X_3$. So that cannot be the marginal of $X_4$. The question is: how do you define the marginal probability of $X_4$? $\endgroup$
    – user227843
    Nov 23, 2018 at 18:31
  • 1
    $\begingroup$ The probability that the trial occurs at all is not conditional as it does not depend on the exact values of the preceeding variables (you can express it as an integral if you like). I'm not sure what else you're looking for. Conditioned on occurrence, the probability of success is $p$ and the probability of occurrence is $P(n /ge k)$. $p$ is therefore the marginal as random variables are implicitly condition on their actual occurrence. You could multiply the two values as well, but this would be a joint probability, not a mariginal. $\endgroup$ Nov 23, 2018 at 18:57
  • $\begingroup$ @deasmhumnha $P(trial | X_1, X_2, X_3) \neq P(trial)$. Also, if I ask you this simple question: what is $P(X_4 = 1)$, unconditionally, what is your answer? $\endgroup$
    – user227843
    Nov 23, 2018 at 19:08
  • $\begingroup$ The process is sequential, so yes, the 4th trial depends on the occurrence of the first three via the common understanding of causality. A marginal distribution must not depend on the particular values of other variables, but is by default predicated on the existence of the system and therefore the existence of preceding variables if generated by a sequential process. So they are in fact equivalent as long as the trial occurs and this follows from independence. The answer is still $p$. $\endgroup$ Nov 23, 2018 at 21:09

2 Answers 2


For context.

$X_4$ can be expressed as the triple $(\Omega, \mathcal{F}, P)$ where $\Omega$ is a sample space, $\mathcal{F}$ is a set of events, sets of zero or more elements of $\Omega$, and $P$ is a measure over events such that $P(\mathcal{\Omega})=1$. For a randomly chosen outcome $\omega \in \Omega$, any event $A\in \mathcal{F}$ such that $\omega \in A$ is said to occur. Clearly, $\Omega = \{0,1\}$ and $\mathcal{F}=\{\varnothing, \{0\},\{1\}, \{0,1\}\}$. We see then by definition of the probability space, the empty set (non-existence) cannot occur; it is outside the scope of the probability space. Therefore all probability spaces require the actual realization of outcomes and $X_4$ is only defined when it is realized.

There are other formulations of probability, but this is the one that dominates statistical theory.

  • $\begingroup$ That's not the correct sample space of the experiment, the sample space is $\{1,1,1\}, \{1,1,0,1\}, \{1,0,0,1,1\}...$ and so on. Or if you want to fill it up with empty sets, $\{1,1,1, \emptyset, \emptyset, \emptyset, ....\}$, $\{1,1,0,1,\emptyset, \emptyset, ... \}$. $\endgroup$
    – user227843
    Nov 23, 2018 at 22:05
  • $\begingroup$ I am referring to $X_4$, as clearly stated. My point is that $X_4$ is only defined where it is realized, thus the marginal is necessarily $p$. $\endgroup$ Nov 23, 2018 at 22:13
  • $\begingroup$ $X_4$ is random variable defined in terms of the sample space, $X_4(\omega)$, where for instance $X_4(\{1,1,0,1\}) = 1$. $\endgroup$
    – user227843
    Nov 23, 2018 at 22:16
  • $\begingroup$ $X_4$ is defined marginally by the sample space of it's outcomes, which are only 0 and 1. You are describing a joint sample space. Honestly, I feel like you're simply trying to contradict every statement I makw, which is fine, but I'm not going to participate any further. My answers remains, take it or leave it. Wait for someone else to give a different take. Your choice. Best of luck. $\endgroup$ Nov 23, 2018 at 22:28
  • $\begingroup$ Your answer is wrong: you are saying that the random variables are independent, and that the marginal of $X_4$ is bernoulli $p$. However if I give you the outcome of 4 coin tosses, and provide you the marginal probabilities of each of them, you will compute the wrong joint distribution by multiplying them together, which proves your answer cannot be correct. $\endgroup$
    – user227843
    Nov 23, 2018 at 22:31

You could use the following sample space, events and probabilities:

Event X1 X2 X3 X4 X5 X6  Prob
3     1  1  1           p^3
4     0  1  1  1        (1-p)*p^3  
4     1  0  1  1        (1-p)*p^3
4     1  1  0  1        (1-p)*p^3
5     1  1  0  0  1     (1-p)^2*p^3
5     1  0  1  0  1     (1-p)^2*p^3
5     1  0  0  1  1     (1-p)^2*p^3
5     0  1  1  0  1     (1-p)^2*p^3
5     0  1  0  1  1     (1-p)^2*p^3
5     0  0  1  1  1     (1-p)^2*p^3
6 .... etc

The marginal probabilities for $X_n$ can then be found by summing the probabilities of the sample space, and for $n>3$ this will be

$$P(x_n) = \begin{cases} P(E<n) &\quad \text{ if } x_n=\emptyset \\ (1-P(E<n))\cdot(1-p) &\quad \text{ if } x_n=0 \\ (1-P(E<n))\cdot p &\quad \text{ if } x_n=1 \end{cases} $$

where $P(E<n)$ is the probability for the event that the number of trials until stopping is smaller than $n$ and can be expressed with the negative binomial.

You could also use:

$$P(x_n|x_n \neq \emptyset) = \begin{cases} 1-p &\quad \text{ if } x_n=0 \\ p &\quad \text{ if } x_n=1 \end{cases} $$

Whichever these two you use is, I guess, up to personal taste which depends on the context and on how you regard the marginal probability and the sample space.

  • $\begingroup$ @statslearner2 This problem seems a bit in danger to become paradoxical when you consider only the two values and not as well the possibility of no value at all. What is the context for this question? $\endgroup$ Dec 3, 2018 at 17:24
  • $\begingroup$ This is actually related to my other questions, I think this approach you are taking points to the right direction. $\endgroup$
    – user227843
    Dec 5, 2018 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.