108 votes
Accepted

XKCD's modified Bayes theorem: actually kinda reasonable?

Well by distributing the $P(H)$ term, we obtain $$ P(H|X) = \frac{P(X|H)P(H)}{P(X)} P(C) + P(H) [1 - P(C)], $$ which we can interpret as the Law of Total Probability applied to the event $C =$ "you ...
tddevlin's user avatar
  • 3,387
38 votes
Accepted

Differences between prior distribution and prior predictive distribution?

Predictive here means predictive for observations. The prior distribution is a distribution for the parameters whereas the prior predictive distribution is a distribution for the observations. If $X$ ...
periwinkle's user avatar
  • 3,363
33 votes

XKCD's modified Bayes theorem: actually kinda reasonable?

Believe it or not, this type of model does pop up every now and then in very serious statistical models, especially when dealing with data fusion, i.e., trying to combine inference from multiple ...
Cliff AB's user avatar
  • 20.7k
19 votes
Accepted

Why LKJcorr is a good prior for correlation matrix?

The LKJ distribution is an extension of the work of H. Joe (1). Joe proposed a procedure to generate correlation matrices uniformly over the space of all positive definite correlation matrices. The ...
Sycorax's user avatar
  • 90.4k
13 votes

Use of Bayesian hierarchical model

In my opinion, there are two different aspects to your question: when should I use a hierarchical model? when should I perform a Bayesian analysis? When should I use a hierarchical model? An ...
jaradniemi's user avatar
  • 4,671
13 votes

Differences between prior distribution and prior predictive distribution?

Let $Y$ be a random variable representing the (maybe future) data. We have a (parametric) model for $Y$ with $Y \sim f(y \mid \theta), \theta \in \Theta$, $\Theta$ the parameter space. Then we have a ...
kjetil b halvorsen's user avatar
12 votes
Accepted

Downsides of inverse Wishart prior in hierarchical models

Here are some relevant resources (full disclosure: the first link is to a paper of mine): http://newprairiepress.org/agstatconference/2014/proceedings/8 http://www.themattsimpson.com/2012/08/20/prior-...
jaradniemi's user avatar
  • 4,671
11 votes
Accepted

What precisely does it mean to borrow information?

This is a term that is specifically from empirical Bayes (EB), in fact the concept that it refers to does not exist in true Bayesian inference. The original term was "borrowing strength", which was ...
Gordon Smyth's user avatar
  • 12.5k
10 votes
Accepted

Is it wrong to use sufficent statistc estimated from the data as a prior in Bayesian data analysis?

Yes, setting $\mu_0=\bar y$ absolutely is circular. The Bayesian model cannot be justified unless you genuinely do have information about $\mu$ that is separate to the data. Based on what you have ...
Gordon Smyth's user avatar
  • 12.5k
9 votes
Accepted

Relation between Bayesian analysis and Bayesian hierarchical analysis?

In my view, hierarchical modeling in a Bayesian setting mainly refers to the building of a complex prior structure. Consider a parameter of interest $\theta_{0}$ and your observation $(x_i)$. Now, ...
beuhbbb's user avatar
  • 5,023
9 votes
Accepted

Use of Bayesian hierarchical model

To the best of my knowledge, there is no opposition between Bayesian models (BM) and hierarchical Bayesian models (HBM) (see e.g. Relation between Bayesian analysis and Bayesian hierarchical analysis?)...
beuhbbb's user avatar
  • 5,023
9 votes
Accepted

Admissible Empirical Bayes Examples

The question has no clear answer because the empirical Bayes formulation does not & cannot specify how the hyperparameter is estimated. Take the simplest Normal mean estimation problem. When$$...
Xi'an's user avatar
  • 104k
9 votes
Accepted

Need some interpretation with plain English for a part in Bayesian Statistics with Beta proability distribution?

In plain English: The Beta distribution family is a set of continuous probability distributions. It describes random variables that can take values anywhere between 0 and 1. One example of a beta ...
eric_kernfeld's user avatar
8 votes
Accepted

How can I apply Bayesian Statistics when the number of data that I have is 1?

TL;DR you can, but the result would strongly depend on your choice of prior. With maximum likelihood, you would be maximizing the likelihood, that in this case is defined in terms of probability mass ...
Tim's user avatar
  • 138k
7 votes

Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters

To demonstrate a solution to this hyperprior problem, I implemented an hierarchical gamma-Dirichlet-multinomial model in PyMC3. The gamma prior for the Dirichlet is specified and sampled per Ted ...
Brad B's user avatar
  • 71
7 votes
Accepted

Comparison between Bayes estimators

First, note that I corrected the original wording of the question wrt the indicator functions in your likelihood definitions as they have to be functions of $x$ not $\theta$. Hence the likelihood is $$...
Xi'an's user avatar
  • 104k
7 votes

What are some statistical tests for exchangeability of a data set?

The theorem in question tells us that exchangeability is equivalent to being conditionally IID. Hence, in practice, data analysts consider the same things when deciding whether observations are ...
Kodiologist's user avatar
6 votes

Comparison between Bayes estimators

Your answer for the squared error loss part is wrong. $$\pi(\theta|x) \propto f(x|\theta) \pi(\theta) = 2\theta x^{\theta-1}I_{(0,1/2)}(\theta). $$ This is a $Beta(\theta,1)$ distribution in $x$, ...
Greenparker's user avatar
  • 15.5k
6 votes
Accepted

Over-parameterization in Bayesian Hierarchical Model

Important to note first: Bayesian inference does NOT automatically guard against overfitting. Adding additional variables will pretty much result in the same problems as in an non-Bayesian analysis. ...
Florian Hartig's user avatar
6 votes

Mixed Effects, Doctors & Operations: predicting on new data containing previously unobserved levels, and updating our confidence accordingly

What you describe is the concept of dynamic predictions from mixed models. Initially, when you have no information for a doctor you only use the fixed effects in the prediction, i.e., you put his/...
Dimitris Rizopoulos's user avatar
6 votes
Accepted

Are analytically tractable posterior distributions exclusively the result of a conjugate relationship in Bayesian hierarchical models?

Conjugate priors are not necessarily tractable (Robert, 1994): take for instance a Beta distribution$$f(x|\alpha,\beta) = B(\alpha,\beta)^{-1} x^{\alpha-1}(1-x)^{\beta-1}\mathbb I_{(0,1)}(x)$$as the ...
Xi'an's user avatar
  • 104k
6 votes

Formulating posterior predictive distribution from hierarchical model

It appears that the stated distribution is for $x_t^{(j)} | \eta_t^{(j)}$ and the random variable $y_{i,t}$ is being ignored for now. It also appears that the authors are being a bit loose in their ...
Ben's user avatar
  • 123k
6 votes
Accepted

Matt's trick (reparametrization) makes my models slower, not faster

It is not unheard of for the centered parameterization to be better. This post on the Stan forums goes into the exact same issue. There it is suggested that [...] centered actually works better when ...
einar's user avatar
  • 4,251
6 votes
Accepted

How to interpret the standard deviation of the slope random effect in a multilevel model

I am assuming you have fit a model in which you have estimated an overall average (fixed) effect of time and allowed the effect of time to vary by region (random slope). The model then estimates the ...
Lachlan's user avatar
  • 1,192
6 votes
Accepted

Is there a difference between hierarchical GAM (HGAM) and Mixed effect GAM (GAMM)?

They are the same thing; we just prefer the terminology "hierarchical" over "mixed", because the salient practical feature of these models is that they can model variation in the ...
Gavin Simpson's user avatar
5 votes

"Unidentified" hierarchical model in brms/stan - where to go from here?

From your model summary I see that you have 4209 obervations in total and 3091 persons. That is, most persons only have 1 corresponding data point and therefore it will be difficult to estimate the ...
Paul Buerkner's user avatar
5 votes
Accepted

Multi-level Bayesian hierarchical regression using rjags

You want a distribution for each quarter (given a state), each state (given a region), and each region. That means you'll need at least some state parameters indexed by s (in your model b0, b1, ...
user98453's user avatar
5 votes
Accepted

How can we convert values proportional to probabilities to Bernoulli probabilities?

Since $p(1)=p$ and $p(0)=1-p$ are both proportional to a known expression* (the unscaled probabilities, $u(i)=c.p(i)$, with the same unknown constant of proportionality, $c$) and you know the $p(i)$ ...
Glen_b's user avatar
  • 281k
5 votes

Ergodicity of MCMC in a hierarchical model

Supose the posterior distribution is denoted by $\pi$ defined on a subset of $\mathbb{R}^d$. Then the Markov chain that samples from this distribution is a general state space Markov chain. Here are ...
Greenparker's user avatar
  • 15.5k
5 votes
Accepted

What problem do these trace plots indicate?

There is probably an issue with your model. Two issues that could lead to such trace plots are: (as mentioned in the comments) An improper posterior distribution. Did you impose proper priors? An ...
Robin Ryder's user avatar
  • 2,076

Only top scored, non community-wiki answers of a minimum length are eligible