Expected distribution of random draws

Question

I have a two part question;

First Part:

I have an urn with 20 balls, 2 of those balls are purple, and I pull out 6 balls at random. I witness 100 realizations of this process.

Given the observed frequency at which I drew purple balls, how do I determine if I am really pulling balls out at random? Also given that there are 2 purple balls, I have a hunch that if purple balls are pulled out disproprionately, I expect that both purple balls would be pulled out (i.e. I'm more interested in seeing if 2 purple balls are pulled out disproportionately than I am if 1 purple ball is pulled out more frequently than expected).

Second Part:

I have an urn with a variable number of balls, a variable number of purple balls within that urn, and a variable number of draws. I witness 100 realizations of this process, and I observe in each realization how many balls there were, how many of those balls were purple, and how many balls I drew from the urn.

Same questions; Given the observed frequency at which I drew purple balls, how do I determine if I am really pulling balls out at random? Again I'm more interested to see if higher frequencies of purple balls are disproportionately drawn than I am to see if 1 purple ball being drawn happens more than expected by chance.

(I'm open to suggestions for title of question and tags)

Edit:

Srikant suggested I may need to make distributional assumptions about my variables, which I am willing to do.

Lets say the number of balls in the urn is uniform between 20 and 30, the number of purple balls is uniform between 0 and 4, and the number of draws is uniform between 6 and 12.

See my answer that describes my motivation for asking this question.

Answer 1

The expected frequency of observing $k$ purple balls in $d$ draws (without replacement) from an urn of $p$ purple balls and $n-p$ other balls is obtained by counting and equals

$$\frac{{p \choose k} {n-p \choose d-k} }{{n \choose d}}.$$

Test a sample (of say $100$) such experiments with a chi-squared statistic using these probabilities as the reference.

In the second case, integrate over the prior distributions. There is no nice formula for that, but the integration (actually a sum for these discrete variables) can be carried out exactly if you wish. In the example given in the edited section -- independent uniform distributions of $n$ from $20$ to $30$ (thus having a one in 11 chance of being any value between $20$ and $30$ inclusive), of $p$ from $0$ to $4$, and of $d$ from $6$ to $12$ -- the result is a probability distribution on the possible numbers of purples ($0, 1, 2, 3, 4$) with values

$0: 69728476151/142333251060 = 0.489896$

$1: 8092734193/24540215700 = 0.329774$

$2: 36854/258825 = 0.14239$

$3: 169436/4917675 = 0.0344545$

$4: 17141/4917675 = 0.00348559$.

Use a chi-squared test for this situation, too. As usual when conducting chi-squared tests, you will want to lump the last two or three categories into one because their expectations are less than $5$ (for $100$ repetitions).

There is no problem with zero values.

---

Edit (in response to a followup question)

The integrations are performed as multiple sums. In this case, there is some prior distribution for $n$, a prior distribution for $p$, and a prior distribution for $d$. For each possible ordered triple of outcomes $(n,p,d)$ together they give a probability $\Pr(n,p,d)$. (With uniform distributions as above this probability is a constant equal to $1/((30-20+1)(4-0+1)(12-6+1))$.) One forms the sum over all possible values of $(n,p,d)$ (a triple sum in this case) of

$$\Pr(n,p,d) \frac{{p \choose k} {n-p \choose d-k} }{{n \choose d}}.$$

Answer 2

First Part: The draws from the urn follow a hypergeometric distribution assuming random draws. Any deviation from the theoretical probabilities vis-a-vis the observed frequencies can be evaluated using chi-square tests.

Second Part:

Let:

$n \sim U(20,30)$ be the total number of balls in the urn

$p \sim U(0,4)$ be the number of purple balls.

$d \sim U(6,12)$ be the number of draws.

$n_{pi}$ be the number of purple balls we the $i^\mbox{th}$ draw.

Thus,

$$P(n_{pi}|n,p,d) = \frac{\binom{p}{n_{pi}} \binom{n-p}{d-n_{pi}}}{\binom{n}{d}}$$

You can integrate the above probabilities using the priors for $n$, $p$ and $d$ which will give you the expected frequencies to observe purple balls provided you draw them at random. You can then compare the expected frequencies with the observed frequencies to assess if the process is truly random.

Answer 3

So here is my motivation for the questions, although I know this is not necessary I like it when people follow up on their questions so I will do the same. I would like to thank both Srikant and whuber for their helpful answers. (I ask no-one upvote this as it is not an answer to the question and both whuber's and Srikant's deserve to be above this, and you should upvote their excellent answers.)

The other day for a class field trip I sat in on the proceedings of an appeals court. Several of the criminal appeals brought before that day concerned issues surrounding Batson challenges. A Batson challenge concerns the use of racial discrimination when an attorney uses what are called peremptory challenges during the voir dire process of jury selection. (I'm in the US so this is entirely in the context of USA criminal law).

Two separate questions arose in the deliberations that were statistical in nature.

The first question was the chance that two out of two asian jurors (the purple balls) seated currently in the venire panel (the urn consisting of the total number of balls) would be selected by chance (the total number of peremptory challenges used equals the number of balls drawn). The attorney in this case stated the probability that both Asian jurors would be selected was $1/28$. I don't have the materials the attorneys presented to the court of appeals, so I do not know how the attorney calculated this probability. But this is essentially my question #1, so given the formula for the Hypergeometric distribution I calculated the expected probability given,

$n = 20$ The number of jurors seated in the venire panel

$p = 2$ The number of Asian jurors seated in the venire panel, both of whom were picked

$d = 6$ The number of peremptory challenges of use to an attorney

which then leads to an expected value of

$$\frac{\binom{p}{p} \binom{n-p}{d-p}}{\binom{n}{d}}=\frac{\binom{2}{2} \binom{20-2}{6-2}}{\binom{20}{6}}=\frac{3}{38}.$$

*note these are my best guesses of the values based on what I know of the case, if I had the court record I could know for sure. Values calculated using Wolfram Alpha.

Although the court is unlikely to establish a bright line rule which states the probability threshold that establishes a prima facie case of racial discrimination, at least one judge thought the use of such statistics is applicable to establishing the validity of a Batson challenge.

In this case there wasn't much doubt in the eyes of the court that Asian jurors had a stereotype among attorneys that they were pro-prosecution. But in a subsequent case a defense attorney used a Batson challenge to claim a prosecutor was being racially biased by eliminating 4 out of 6 Black women. The judges in this appeal were somewhat skeptical that Black women were a group that had a cognizable stereotype attached to them, but this again is a question amenable to statistical knowledge. Hence my question #2, given 100 observations could I determine if black women were eliminated using peremptory challenges in a non-random manner. In reality the attorneys are not eliminating prospective jurors based on only the race and sex of the juror, but that would not preclude someone from determining if the pattern of peremptory challenges at least appears or does not appear random (although non-randomness does not necessarily indicate racial discrimination).

Again I'd like to say thank you both to whuber and Srikant for their answers.