1
$\begingroup$

I'm trying to understand the distribution of a normalized sum of a product of Bernoulli random variables. Specifically, I have two vectors $M_1$ and $M_2$ of length n. Each element of each vector is a Bernoulli random variable with varying success probability $p_i$. We can assume for now that each element is independent from all other elements and the vectors are independent from each other. With this assumption the sum of elements of vector $M_1$ is $\sum_i M_{1i}$ and follows a Poisson binomial distribution with expected value $\sum_i p_{1i}$. The element-wise product of both vectors $M_1 M_2$ consists again of Bernoulli random variables, this time with success probabilities $p_{1i}p_{2i}$ for each element. The sum of this vector of products $\sum_i M_{1i}M_{2i}$ (which is also the inner product of the two vectors) is then again distributed with a Poisson binomial distribution with expected value $\sum_i p_{1i}p_{2i}$. What I am interested in now is the distribution of $Z=\frac{\sum_i M_{1i}M_{2i}}{\sum_i M_{1i}}$. Basically this is the number of common successes between the two vectors divided by the number of successes in the first vector. This problem seems simple enough to possibly have a closed-form expression, but I don't know what to look for. Assumptions of n being large are acceptable, but exact solutions are preferred. Any pointers to either a solution or a problem of a similar form in an application would be greatly appreciated.

$\endgroup$
7
  • $\begingroup$ If by "closed-form" you mean an expression that asymptotically in $n$ is $o(n)$ in length, you will necessarily be disappointed because it depends (at a minimum) on all the $p_{2i}$ separately. Approximations can be developed by making assumptions about the average relationships among the $p_{i1}$ and $p_{2i},$ so if you have such information, please share it. $\endgroup$
    – whuber
    Commented Jan 28 at 19:25
  • $\begingroup$ thank you for the clarification. I'm fine if the expression depends on all the p_{2i}, especially if like in the Poisson Binomial Distribution it would result in a straightforward expected value and variance. Even better of course would be if it converges to a well-known distribution. On the average relationship between p_{1i} and p_{2i} : I don't think there is anything else but to say that they are conditionally independent from each other. But would specifying P(p_{2i}|p_{1i}) be helpful? I might be able to make an assumption based on what my data shows $\endgroup$
    – Nils R
    Commented Jan 31 at 15:52
  • $\begingroup$ Convergence results likely depend on details of what happens to the $p_{ji}$ as $i$ grows large. $\endgroup$
    – whuber
    Commented Jan 31 at 15:54
  • $\begingroup$ p_{ji} would not change in distribution as i gets large, in the sense of probabilities becoming smaller or larger, if that's what you mean. $\endgroup$
    – Nils R
    Commented Jan 31 at 15:58
  • $\begingroup$ Are you supposing these values are themselves random variables? If so, that would be useful information to include -- along with any assumptions about those distributions. $\endgroup$
    – whuber
    Commented Jan 31 at 16:25

1 Answer 1

0
$\begingroup$

Here is my attempt at the distribution; more elegant/compact solutions quite possibly exist however.

Let $A:=\sum_i M_{1i}$ and $B:=\sum_i M_{1i}M_{2i}$. Also, suppose for some given $A$, let $M_1^{A,k}$ correspond to a specific $M_1$ vector whose sum is $A$. So, there are ${n\choose A}$ different $M_1^{A,k}$ vectors for a given value of $A$.

The joint PMF $P(A=a,B=b)$ where $b\leq a\leq n$ is \begin{align*} P(A,B) &= P(A)P(B|A)\\ &= P(A)\sum_kP(B|M_1^{A,k})P(M_1^{A,k}|A)\\ &= \sum_kP(B|M_1^{A,k})P(M_1^{A,k}) \end{align*} Now, $P(B|M_1^{A,k})$ is a Poisson-binominal PMF. Specifically, suppose $\{j\}$ is the set of indices of the "successful" components of $M_1^{A,k}$. We then look at $M_2$ and focus on its $j$ components as well; $P(B|M_1^{A,k})$ is simply the probability that $B$ of those $M_{2j}$ components are successful.

Now, we look at $P(M_1^{A,k})$, which is simply the probability of having the $j$ components be the successful ones, ie $P(M_1^{A,k})=\prod_{i\in\{j\}}p_{1i}\cdot \prod_{i\not\in\{j\}}(1-p_{1i})$

Now that we have expressions for our summands in the joint PMF $P(A,B)$, we turn to $Z=\frac{B}{A}$. Clearly $Z$ is discrete and rational-valued. Its PMF will be $$P(Z=z)=\sum_{(a,b)\;st.\;b/a=z,\; b\leq a\leq n }P(A=a,B=b)$$

Technically that's it, and it's "closed-form" given all the sums/products are finite. But it's ugly sorry!

$\endgroup$
2
  • $\begingroup$ Thank you for your answer, this is really helpful with notation and with the conditional probability idea, will +1 once I have enough reputation. I won't accept it yet hoping that someone will come up with a smart assumption that can lead to a nice closed-form solution or at least approximation. Could you think of a similar problem to this which does have a known distribution as the solution? Maybe I could take some inspiration from there $\endgroup$
    – Nils R
    Commented Jan 31 at 16:49
  • $\begingroup$ @NilsR I imagine things will simply greatly if further assumptions or restrictions are placed on the $p_{ji}$ values; otherwise, I fail to see how we can avoid long sums or products. The other thing of course is that $Z$ is rational, so its PMF will likely exihibit lots of "unsmoothness", which is not conducive to the existence of a compact PMF (which is likely to be continuous if extended to the reals). $\endgroup$ Commented Jan 31 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.