Which Bayesian update is the right one to use in this case? Suppose I have many widgets, and a widget can be good or bad. I can test widgets in batches. The batch is good if all widgets in the batch are good, and it is bad if any widget in the batch is bad. A widget can be tested multiple times, as part of different batches.
For every widget, I have a probability that that particular widget is good. How should I update that probability when the test result of a new batch comes in?
When a batch is good, all widgets in that batch are good, we can set their probability to 1. When a batch $B$ is bad, my initial attempt for updating widget $w$’s probability was:
$$
\mathbb{P}(w \textrm{ is good } \mid B \textrm{ is bad})
= \frac{
  \mathbb{P}(B \textrm{ is bad} \mid w \textrm{ is good})
  \mathbb{P}(w \textrm{ is good})
}{
  \mathbb{P}(B\textrm{ is bad})
}
= \frac{
  \left(1 - \prod_{v \in B, v \neq w} \mathbb{P}(v \textrm{ is good})\right)
  \mathbb{P}(w \textrm{ is good})
}{
  1 - \prod_{v \in B} \mathbb{P}(v \textrm{ is good})
}
$$
Note, the end result depends on the order in which we test batches. For example, suppose we start with a prior probability of 0.9 for each widget, and then we learn that batch $B = \{w_1, w_2\}$ is bad. Afterwards we have probabilities $(0.47, 0.47, 0.9)$ of being good. Then we learn that batch $C = \{w_1, w_2, w_3\}$ is bad. Afterwards we have probabilities $(0.34, 0.34, 0.87)$. But suppose we tested $C$ first and then $B$. Then we would have gone to $(0.63, 0.63, 0.63)$ and then to $(0.39, 0.39, 0.63)$.
On the one hand, this makes complete sense to me. If we already suspect that $w_1$ or $w_2$ is bad, it’s no surprise that $C$ is bad, so we shouldn’t update as much upon learning that, as the other way around. On the other hand, the order can be arbitrary, I don’t want the outcome to depend on it.
Consider learning that a widget is bad: we start with a prior probability of 0.9 for each widget, we learn that batch $B = \{w_1, w_2\}$ is bad, so we update the probabilities for $w_1$ and $w_2$ to 0.47, but then we test $w_1$ in isolation and conclude with certainty that it is bad. Doesn’t that mean that $B$ was likely bad due to $w_1$, and we should be less harsh on $w_2$? If we learn that $w_1$ is bad after learning that $B$ is bad, we think $w_2$ only has 0.47 probability of being good, but if we learned that $w_1$ was bad first, then we would keep it at 0.9.
Consider learning that a widget is good: we start with a prior probability of 0.9 for each widget, we learn that batch $B = \{w_1, w_2, w_3\}$ is bad, so we update the probabilities for $w_1, w_2, w_3$ to 0.63, but then we test $w_1$ in isolation and conclude with certainty that it is good. Shouldn’t we be harsher on $w_2$ and $w_3$ then? If we learn that $w_1$ is good after learning that $B$ is bad, we think $w_2$ and $w_3$ have 0.63 probability of being good, but if we learned that $w_1$ was good first, then we would lower it to 0.47.
One thing I might do is to keep track of all bad batches, and after I learn that a batch is good, re-evaluate all evidence of bad batches from scratch, but remembering which widgets are good. Or in other words, evaluate all the evidence for good batches before evidence for bad batches. But the outcome still depends on the order of the bad batches.
An alternative I considered is to start from scratch every time, and aggregate all the evidence in a single update:
$$
\mathbb{P}(w \textrm{ is good } \mid B_1, \ldots, B_n \textrm{ are bad})
= \frac{
  \mathbb{P}(B_1, \ldots, B_n \textrm{ are bad} \mid w \textrm{ is good})
  \mathbb{P}(w \textrm{ is good})
}{
  \mathbb{P}(B_1, \ldots, B_n \textrm{ are bad})
}
$$
This removes the dependence on the order of the batches. But we also have a choice: do we update the probability for every widget at the same time, from the same prior? Or do we update widget $w$ first, and use its posterior probability when updating widget $v$? Here at least I think that sequential updates are wrong, because in a sense they apply the same evidence multiple times.
How can I decide which of these different ways of computing a posterior probability is the “right” one? Or is there maybe another approach that makes more sense to use?
For additional context, for my use case, the batches and their test results are not known in advance. I actually want to use the probabilities to construct batches that are likely to be good.
 A: Let
\begin{equation}
w = (w_1,\ldots,w_n) ,
\end{equation}
where $w_i \in \{0,1\}$ and $w_i = 1$ indicates widget $i$ is good and $w_i = 0$ indicates it's bad. Note $w \in \mathcal{W} = \{0,1\}^n$
Let $B_j$ be the outcome of a "batch test" that depends on a subset of the widgets: $w^j \subset w$ (treating $w$ as a set for this purpose).
In particular, let
\begin{equation}
B_j = \prod_{w_i\in w^j} w_i . 
\end{equation}
In other words, if all of the widgets in the batch are good then $B_j = 1$  and otherwise $B_j = 0$. Let $w_\star^j$ denote the value of $w^j$ that consists of all good widgets:
\begin{equation}
w_\star^j = (1, \ldots, 1) .
\end{equation}
Then we can write (spend a moment on this)
\begin{equation}
p(B_j|w^j) = \begin{cases}
B_j & w^j = w_\star^j \\
1-B_j & w^j \ne w_\star^j
\end{cases} .
\end{equation}
In addition we have
\begin{equation}
p(w^j = w_\star^j) = p(\text{$w_i = 1$ for all $w_i \in w^j$}) .
\end{equation}
Bayes rule provides the posterior distribution for $w$:
\begin{equation}
p(w|B_j) = \frac{p(B_j|w)\,p(w)}{\sum_w p(B_j|w)\,p(w)}
\end{equation}
Since the likelihood only depdends on $w^j$, we can express the posterior for $w$ as
\begin{equation}
p(w|B_j) = p(w^j|B_j)\,p(w^{-j}|w^j) ,
\end{equation}
where $w^{-j} = w\setminus w^j$ and
\begin{equation}
p(w^j|B_j) = \frac{p(B_j|w^j)\,p(w^j)}{\sum_{w^j} p(B_j|w^j)\,p(w^j)} .
\end{equation}
If the elements of $w$ are a priori independent then $p(w^{-j}|w_j) = p(w^{-j})$.
Sequential batch tests
Given two different batch tests we can write
\begin{equation}
\begin{split}
p(w|B_1,B_2) &= \frac{p(B_1,B_2|w)\,p(w)}{p(B_1,B_2)} = \frac{p(B_2|w^2)\,p(B_1|w^1)\,p(w)}{p(B_2|B_1)\,p(B_1)} \\
&= \frac{p(B_2|w^2)\,p(w|B_1)}{p(B_2|B_1)} .
\end{split}
\end{equation}
Similarly we can write
\begin{equation}
p(w|B_1,B_2) = \frac{p(B_1|w^1)\,p(w|B_2)}{p(B_1|B_2)} ,
\end{equation}
which shows that it doesn't matter what the order of the batch tests is.
Example
As an example, suppose $n = 3$ and let $p(w) = p(w_1)\,p(w_2)\,p(w_3)$ where $p(w_i = 1) = \theta$. Consider two batch tests: $B_1(w_1,w_2) = 0$ and $B_2(w_1,w_3) = 0$.
Here are the posterior probabilities for the 8 values for $(w_1,w_2,w_3)$:
\begin{gather}
p(0,0,0) = \frac{(\theta -1)^2}{-\theta ^2+\theta +1}, \quad
p(0,0,1) = \frac{(\theta -1) \theta }{\theta ^2-\theta-1}, \quad
p(0,1,0) = \frac{(\theta -1) \theta}{\theta ^2-\theta -1}, \quad \\
p(0,1,1) = \frac{\theta ^2}{-\theta ^2+\theta+1}, \quad
p(1,0,0) =  \frac{(\theta -1) \theta}{\theta ^2-\theta -1}, \\
p(1,0,1) = 0, \quad
p(1,1,0) = 0, \quad
p(1,1,1) = 0 .
\end{gather}
