# Density function for a multivariate Bernoulli-like distribution

I'm looking for a distribution to model a vector of $k$ binary random variables, $X_1, \ldots, X_k$. Suppose I have observed that $\sum_i X_i = n$. In this case I do not want to treat them as independent Bernoulli random variables. Instead, I would like something like the multinomial:

$P(X_1=x_1, \ldots, X_k=x_k) = f(x_1, \ldots, x_k; n, p_1, \ldots, p_k) = \frac{n!}{x_1! \cdots x_k!} \prod_{i=1}^k p_i^{x_i}$

but instead of the $x_i$ being nonnegative integers, I want them restricted to be either 0 or 1. I have been trying to see if the multivariate hypergeometric is appropriate, but I'm not sure.

-
By the way, will the binomial not work for you? You have k trials and n successes. right? –  user28 Aug 1 '10 at 23:16
Slightly different: I have $k$ different coins, each with a different probability (ie. $p_i$), and I have $n$ successes. Sorry for the confusion! –  Christopher DuBois Aug 2 '10 at 1:43

The appropriate distribution is Wallenius's noncentral hypergeometric distribution. Using an urn analogy, the problem is equivalent to picking $n$ of $k$ balls without replacement, where each ball is a different color: the parameters $p$ are analogous to the weights of picking a particular color.

The problem: it's not very convenient to work with, though there is an R package.

-
Interesting, thanks. I wonder though if it's not the Fisher that's more appropriate here -- since in the urn formulation of the problem, N is observed after the experiment, and you condition on the sum. Quite likely, I'm one of the "confused" the wikipedia article refers to. :-) –  ars Aug 2 '10 at 23:52

Update

Approach 1: Difficult to implement/analyze

Consider the simple case of $k$ = 3 and $n$ = 2. In other words you toss 3 coins (with probabilities $p_1$, $p_2$ and $p_3$). Then, the required mass function for the above case is:

$p_1 p_2 (1-p_3) + p_1 (1-p_2) p_3 + (1-p_1) p_2 p_3$

The above reduces to the binomial if the probabilities $p_i$ are all identical.

In the general case, you will have ${k \choose n}$ terms where each term is unique with a structure similar to the one above.

Approach 2: Easier to analyze/implement

Instead of the above, you could model each $X_i$ as a bernoulli variable with probability $p_i$. You could then assume that $p_i$ follows a dirichlet distribution.

You would then estimate the model parameters by constructing the posterior distribution for $p_i$ conditional on observing $n$ successes.

If you can normalize by n and and assuming that treating them as probabilities/proportions makes sense in your context you can use the dirichlet distribution.

-
I am already modeling the probability of each cell as a Dirichlet, $p \sim Dirichlet(\alpha)$. Normalizing by $n$ wouldn't quite work since a cell with large $p_i$ would have small likelihood whenever $n$ is fairly large, even when we indeed observed $x_i=1$ (unnormalized). Does that make any sense? –  Christopher DuBois Aug 1 '10 at 23:08
I am afraid you lost me by the mention of a 'cell' when the question does not mention this word at all. Could you provide some more context so that I can understand better? –  user28 Aug 1 '10 at 23:13
Sorry: By cell I mean each $x_i$. –  Christopher DuBois Aug 1 '10 at 23:14
In other words, I want to sample $n$ integers between 1 and $k$ without replacement using the probability vector $p$. –  Christopher DuBois Aug 1 '10 at 23:17
This getting confusing to me. Your $X_i$ is either 0 or 1 but now you want to sample $n$ integers between 1 and $k$? –  user28 Aug 1 '10 at 23:23