Assuming we are doing a task to find how runs will be scored some match and for that, we have assumed some prior, now what if we find out midway through the process on which we built our priors was inaccurate due to various factors. How should we the strategize for the further process and at the same time not dump the previous work done so far though the initial priors were not correct?
5
-
$\begingroup$ Prior distributions are often deemed inaccurate once one has seen data. However, sufficient data will overwhelm most priors and produce satisfactory posterior distributions. $\endgroup$– BruceETCommented Jun 27, 2020 at 21:56
-
$\begingroup$ Thank you Sir, Suppose if somehow we want to use that prior information then what strategy should we use ahead? $\endgroup$– Kabir KCommented Jun 27, 2020 at 22:02
-
1$\begingroup$ One usually tests the effect of different priors on the inference of the model. It's called sensitivity analysis. If your model is very sensitive to the choice of the prior, or there is a prior you choose for the final model, then these aspects must be discussed and justified. $\endgroup$– user289381Commented Jun 27, 2020 at 22:09
-
1$\begingroup$ This thread is perhaps somewhat related: stats.stackexchange.com/q/86472/77222 $\endgroup$– Jarle TuftoCommented Jun 27, 2020 at 22:22
-
1$\begingroup$ @KabirK. One does not 'adjust' a prior based on preliminary information from planned data collection. That would be considered pointless or dishonest or both. See my Answer. // A bad, weak prior may be overwhelmed by data. Also, one can do a 'second round' with the old posterior as the new prior pending additional data. $\endgroup$– BruceETCommented Jun 27, 2020 at 22:46
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Suppose you are trying to predict the outcome of an election on Proposition A, which needs 60% Yes votes to pass. Let $\theta$ be the unknown proportion of the electorate in favor.
Maybe you have a favorable view of the probability of success, so you choose the prior $\mathsf{Beta}(7,3),$ which implies $P(\theta > 0.6) = 0.77$
1 - pbeta(.6, 7, 3)
[1] 0.768213
You discover that early results from a public opinion poll are running only about 50-50, and you begin to have doubts about your prior distribution. There are a couple of possibilities (among many):
- Your prior is grossly wrong.
- Your prior is OK and the early returns from the poll are not yet reliable.
In either case, you should just wait for the final poll results.
If they show 735 in favor out of 1000, then the likelihood function is proportional to $\theta^{735}(1-\theta)^{265},$ the beta posterior distribution is $\mathsf{Beta}(742,268),$ a 95% Bayesian interval estimate for $\theta$ is $(0.71, 0.76),$ your prior doesn't look so bad in retrospect, and the proposition looks on target for success.
qbeta(c(.025,.975), 742, 268)
[1] 0.7070060 0.7614208
Alternatively, the final poll results may show only 520 out of 100 in favor, then the posterior distribution is $\mathsf{Beta}(527,483),$ a 95% Bayesian interval estimate for $\theta$ is $(0.49, 0.55),$ the data have overwhelmed your less-that-ideal prior, and it seems proponents of Proposition A have some work to do before election day.
qbeta(c(.025,.975), 527, 483)
[1] 0.4909563 0.5525263
Of course, it is possible, if ordinarily imprudent, to begin with such a strong prior that only a huge amount of data could overwhelm it. Also, if the people determining the prior distribution are truly clueless about Proposition A and its prospects, then they might admit their indecision and give a relatively noninformative prior such as $\mathsf{Unif}(0,1) \equiv \mathsf{Beta}(1,1).$ In that case the prior distribution will have almost no effect on the eventual prediction interval.
Finally, if it seems for some reason that the first probability interval estimate is not adequate, then the posterior distribution might become the prior distribution for another round of Bayesian estimation based on a subsequent poll.
