Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

As I understand it, selecting a prior provides something of a starting point for your analysis. From there, the distribution is shaped by the observed data. Obviously, the more data you observe, the more discrepancy is possible between the prior and posterior distributions (especially if the selected prior is inappropriate). As a result, it would seem to make sense that, for some large n, the selection of a prior is essentially irrelevant because the observed data will overwhelm the prior. Is this, in fact, the case? If so, does this actually occur in practice (or does that value of n need to be so ridiculously large that the point is purely theoretical)?

The underlying problem that I’m facing is that if I have m data points and I’m concerned about the appropriateness of my prior, what are some tools at my disposal to determine if my concern is legitimate?

Note: I realize that this question is very theoretical and that a concrete answer isn’t really possible (I’m sure a lot of this depends on the types of distributions, how inappropriate the prior is, etc.), so I’m worried that this might violate the condition that questions must be “practical, answerable questions based on actual problems you face.” If this is the case, please let me know. I’m new to the site and don’t really have a firm grasp on the etiquette yet…

share|improve this question
    
You are right that the prior becomes irrelevant when you have large $n$. What is the sample size here? –  Macro Jan 16 '13 at 14:57
    
@Macro In the ballpark of 10,000. –  Kyle Wurtz Jan 16 '13 at 15:03
1  
I strongly suspect that, with a sample size like that, the prior will have almost no influence. An exception to this would be if there are feasible areas of the parameter space where the prior has 0 mass. I'd suggest trying different priors and seeing how your estimator changes as a way to empirically see how much it matters. –  Macro Jan 16 '13 at 15:05
    
Sensitivity testing definitely makes sense, and I suppose it's probably the best bet given the variability in scenarios (your exception regarding 0 mass). Thanks for your input! –  Kyle Wurtz Jan 16 '13 at 15:18
add comment

1 Answer 1

up vote 6 down vote accepted

It's always possible to create a prior that will overwhelm your data, no matter how many observations you have. However, for any fixed prior, as the number of observations grows, the influence of the prior shrinks (except for the 0-mass case that Macro pointed out in his comment).

For some prior distributions there's a concept of "prior sample size": if your prior sample size is $n_p$ and you have $n$ observations, then the posterior is in some sense a weighted average of the prior and data, weighted with $n_p$ and $n$ respectively. The easiest place to see this is when the Beta distribution is used as a prior for the Binomial distribution, where the prior sample size is $\alpha+\beta$. If I use a $\operatorname{Beta}(4,1)$ prior, that's sort of like saying that I believe my prior information is as good as 5 observations, and I expect success 80% of the time. If I then observe 5 data points (say 3 successes, 2 failures) my posterior will then be $\operatorname{Beta}(7,3)$--now my posterior is worth 10 observations (5 prior + 5 data), with a mean of .7. The prior is still pretty strongly weighted here. But if I observe 500 observations then my prior is basically irrelevant, because my data sample size is 100 times as large as my prior sample size.

On the other hand, I could use a $\operatorname{Beta}(8000,2000)$ prior. In this case, even if I observed 5000 data points, my posterior is still mostly determined by my prior.

If you're in a case where it's easy to calculate this sort of "prior sample size" (which also includes common models such as Normal-Normal, InverseGamma-Normal, and Gamma-Poisson), then this can give you an idea of how influential your prior is relative to your data. Otherwise I try to err on the side of diffuse priors, on the basis that it's (usually) better to overestimate your posterior uncertainty than to underestimate it.

share|improve this answer
    
This was a very helpful answer! The concept of the prior and the data creating something of a weighted average was the idea that was bouncing around in my head, but your examples helped solidify that concept. Thanks! –  Kyle Wurtz Jan 16 '13 at 21:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.