How to do bayesian updating when a statistical test gives insignificant results? Consider an outcome variable Y that you want to maximize. To do that you need to choose between two "treatments" A and B. 
To find which to choose, you run an experiment that gives as a result that y under A is larger than y under B. The difference, however, is not statistically significant. You conclude that the measured effect is not systematic but due to randomness. So, I guess, the experiment does not help you make a choice as you should be still indifferent between A and B. 
This conclusion, however, contrasts with my intuition that, after the experiment, one should update the initial beliefs about the effectiveness of A and B. Under this view, the experiment is indeed informative and you are not indifferent between A and B (A should be preferred to B), even though the difference is not statistically significant.
Is this a true paradox? Or just a mistake in my reasoning?
In other words, after the experiment, should I consider A and B equal and flip a coin, as statistical significance seems to suggest? or A is better than B as (Bayesian) updating seems to suggest? 
 A: In the first scenario you gather some data and conduct the hypothesis test, the test tells you that the difference is so small that it didn't reach statistical significance.
In the second scenario, you start with some hypothesis (your priors), gather the data, and combine the two sources of information, to update your hypothesis. Bayesians don't use $p$-values, so they won't say that the result is "not significant", yet they can simply conclude that it is small. On another hand, Bayesian could define some region of practical equivalence and conclude that since the small difference falls into the region, it is "practically equivalent" to zero (call it "insignificant" if you wish).
There is no paradox in here. Both approaches would only differ in the conclusions if the Bayesian used strongly informative prior, that would influence the results, but if the priors are not "too" informative and the data is not too "inconclusive", this should not be the case.
A: The mistake in your reasoning is "You conclude that the measured effect is not systematic but due to randomness". The measured effect is small so after the experiment it's not clear if that small effect is due to randomness or if it is due to a small systematic effect. If you increase your sample size you could detect smaller systematic effects and be confident that they are not due to randomness
