4
$\begingroup$

The question in the title is usually discussed subjectively, but I'm wondering if there is a way to answer it with some mathematical rigor...

Suppose I have experiment A that generated data $x_1$ and later I do experiment B and collect data $x_2$. Is it fair to say that an informative prior developed from $x_1$ should be used to analyze $x_2$ if and only if I consider the combined data $(x_1, x_2)$ exchangeable?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Very interesting question. I doubt there is a definite answer to this, but let me give an extended comment. Imagine that you've done an experiment on some phenomenon that was never studied by anyone before. During your analysis, as a Bayesian, you would choose some prior for analyzing this data. Since you have no prior knowledge, you choose some "uninformative" prior. Next, given the results, you are planning a follow-up experiment, to study some further aspects of the problem. The design of the follow-up experiment would depend on the previous results (they are not exchangeable!), so should you ignore the previous results and again use "uninformative" prior, or rather consider them when choosing your prior? I would say that ignoring the results does not sound as a good idea.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Well, suppose we can't update the design of experiment B based on experiment A because there are no factors. Or, if there are factors, because we intend to do a full factorial design on the same factors and levels in both experiments A and B. $\endgroup$
    – JTH
    Commented Aug 23, 2019 at 20:38
  • $\begingroup$ Example: Machine A is old and known to make good widgets that have a $p_0 = 90\%$ probability of not breaking. We have data from a past test that indicates this. Machine B is a brand new update of Machine A that is required to also make widgets that have a $90\%$ chance of not breaking. Machine B is supposed to be better than Machine A, but has been completely redesigned. The makers of Machine B argue that we don't have to test Machine B very much because we can just use the data from Machine A to form a prior on $p$. My inner Bayesian agrees, but it seems risky to subject B to less testing. $\endgroup$
    – JTH
    Commented Aug 23, 2019 at 20:50
  • $\begingroup$ My thought is that the quality of widgets from Machine A and B are not exchangeable, so an informative prior from Machine A cannot be used to asses Machine B. $\endgroup$
    – JTH
    Commented Aug 23, 2019 at 20:56
  • 1
    $\begingroup$ There are many ways to come at this. But if you can learn anything useful from Machine A -- and by useful I mean transferable to your understanding and expectation for Machine B -- then you should not ignore what you learn from Machine A. If Machine A uses different technology, comes from a different vendor, has been used by a different operator, and otherwise seems totally unlike Machine B, then perhaps the only transferable information is the fact that successful production is not impossible in principle. Surely, that is useful. $\endgroup$
    – Peter Leopold
    Commented Aug 23, 2019 at 21:27
  • $\begingroup$ Is there any guidance of how to determine a prior that is "between" uninformative and highly informative? Suppose the test on Machine A produced $10,000$ widgets. The prior is "overwhelmingly informative" in my opinion, and new observations from Machine B might not affect it very much. $\endgroup$
    – JTH
    Commented Aug 23, 2019 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.