Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I am trying to see how different priors affect the posterior estimates in bayesglm{arm}. But no matter what the prior is, why is the posterior hardly changing?

library(Zelig); data(turnout) #Using the "turnout" data from the "Zelig" package

library(arm) #Loading the "arm" package
model.1 = bayesglm(vote ~ race + age, family=binomial(link=logit),
                   prior.df=1, prior.scale=2.5,
                   data=turnout) #This is the default specificication with a weakly informative prior

summary(model.1) yields:

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) 0.042320   0.176567   0.240  0.81058    
racewhite   0.641959   0.134136   4.786  1.7e-06 ***
age         0.011222   0.003046   3.683  0.00023 ***

Now compare this to:

model.2 = bayesglm(vote ~ race + age, family=binomial(link=logit),
                   prior.df=1, prior.scale=2.5, prior.mean=c(-100,0), #Specifying a crazy prior
                   data=turnout)

Summary(model.2) yields:

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) 0.039891   0.176720   0.226 0.821410    
racewhite   0.645076   0.134465   4.797 1.61e-06 ***
age         0.011219   0.003047   3.682 0.000231 ***

What's going on here? Surely a prior which is so different would influence the maximum likelihood estimates enough to sharply change the posterior?

share|improve this question
1  
what is your sample size? I'm not familiar with this R package but the prior becomes less and less important as the sample size increases - here is a question and two answers related to that. – Macro Aug 1 '12 at 14:10
Your data has 2000 points. If the data has high precision and large N this is what I would expect. If you sample the data down to just 50 observations, the estimates change more with different priors. – Seth Aug 1 '12 at 14:12

migrated from stackoverflow.com Aug 1 '12 at 14:02

1 Answer

up vote 4 down vote accepted

Your second prior is not quite so crazy as you think. A t-distribution with one degree of freedom has very wide tails. In this with location = -100 and scale = 2.5 you still have a density of > 1e-5 for 0.645 and 0.0112. This is still more than 1/2000 of the maximum density of your prior. This is easily overwhelmed by the likelihood function for your amount of data. BTW the 1/2000 is not intended to create a link to your sample size, i just want to made concrete how wide the tails of your prior are.

share|improve this answer
Relevant reference: A. O'Hagen, 1990. Outliers and credence for location parameter esimation. JASA 85(409), 172-176. Short, short version: it's not the absolute value of the prior density per se that matters but rather how flat the prior density is in the region of high likelihood, and vice versa. – Cyan Aug 1 '12 at 18:53
That makes sense. Thanks, Erik and all. Cyan... thanks especially for the citation. – user3671 Aug 2 '12 at 3:51
Secondary question: Is there any way to figure out how off the prior mean would have to be, given a reasonably wide distribution near the mode? I'm sure there's going to be a mathematically way of doing it. I'd be more interested in simulation/graphical demonstration. – user3671 Aug 2 '12 at 4:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.