The ambidextrous policeman problem Consider an ambidextrous policeman who wants to decide with which hand he should shoot by default (right, R, or left, L).
As evidence for his decision he performs a serious of controlled experiments B1, B2, etc (e.g., a shootout in the street, in a closed space, at night, etc) to obtain likelihoods of hitting the target with a particular hand: p(B1|R) vs p(B1|L), p(B2|R) vs p(B2|L), etc.  
If shooting was the only tool of his trade, and assuming the experiments to be independent, his decision would be based on evaluating:
p(R|B)/p(L|B) = [p(B|R)/p(B|L)]*[p(R)/p(L)] 
with p(B|R) = p(B1|R)p(B1|R)... and p(B|L) = p(B1|L)p(B1|L)..., given a specification of prior beliefs p(R) and p(L).
However, to catch criminals the policeman has other options at his disposal besides shooting: he can call for back-up, run, drive a car, etc. To see how counter-intuitive the application of the preceding formula is consider that
p(B1|R) = 0.9, p(B1|L) = 0.8, p(B2|R) = 0.1, p(B2|L) = 0.2
In this case the likelihood ratio is 0.5625, thus suggesting that he should use the left hand. However, this result is driven by the second experiment, in which the probability of hitting is very low... hence suggesting that the policeman will do something else other than shooting if faced with that situation. But on the other hand simply ignoring experiment B2 seems too radical. The decision method should weigh the different empirical observations on the basis of how likely the policeman is to use a gun as opposed to do something else.
Can you suggest an alternative approach?
I am not a professional statistician but this problem arose in my work. I want to select a model out of a population of models which is not exhaustive, using a set of (non-controlled) empirical observations. Somehow it is necessary to weigh the observations such that those in which all models under consideration perform poorly affects the decision proportionately less than observations in which they perform well.
All help appreciated.  
 A: You said which hand should he shoot with "by default", which I take to mean that he is going to choose one hand for all situations. In that case, he should compute his overall hit chance with each hand $P(\text{Hit}\,|\,L)$ and $P(\text{Hit}\,|\,R)$ and choose the hand that gives him the greater value. To do this, he must consider all possible situations he may encounter and estimate his hit chance in each situation, then weight it by the chance of being in that situation. Let $H \in \{L,R\}$ represent the possible hand choices, and $B_1, B_2, \ldots, B_n$ the set of all situations he may encounter. Then the policeman can compute his overall hit chances using the law of total probability:
$$P(\text{Hit}\,|\,H) = \sum_{i=1}^n P(\text{Hit}\,|\,H,B_i)P(B_i)$$
(Here I have implicitly assumed that $P(B_i\,|\,H) = P(B_i)$, that is, the policeman's probability of being in a situation is not affected by his hand choice.) Now we just use the policeman's shooting experiments to estimate the conditional hit probabilities $P(\text{Hit}\,|\,H,B_i)$, and to estimate $P(B_i)$ we do statistics on his work history to determine what situations he tends to be presented with.
It is not clear to me how it is relevant that the policeman may choose to do different things besides use a gun. What the policeman needs to know is, in those situations which he would use a gun, what is the best hand to use? We can formalize this by modifying the formula above, conditioning the events on the choice $C$ to use a gun:
$$P(\text{Hit}\,|\,H, C=\text{Gun}) = \sum_{i=1}^n P(\text{Hit}\,|\,H,B_i,C=\text{Gun})P(C=\text{Gun}\,|\,B_i)P(B_i)$$
As before, all the terms in this formula must be determined empirically either by experimentation or an examination of the policeman's work history. 
At no point do I see any reason to use Bayes' rule in this problem, as you are doing in your first equation. Bayes' rule would only be needed if the policeman had not collected enough data to determine his hit probabilities in different situations, and needed to combine his experimental data with some kind of prior hunch of his accuracy in each situation.
