If data is generated from a distribution whose parameters is a random variable, can the parameters' distribution be recovered using bayesian methods?

Question

I'm a Computer Science student currently studying Bayesian Statistics and I'm doing some simple simulations in R to become more familiar with the concepts.

Recently, I tried generating samples of data $X$ where $X|p\sim Be(p)$ and $p\sim \text{Unif}(0.25,0.4)$. This was done by generating random samples of $p$ and then using each random sample of $p$ to generate 1 sample of $X$.

I then used a uniform prior on $[0,1]$ and tried to recover the distribution of $p$ using Bayesian methods. But I ended up with a posterior distribution that has the bell curve-like shape centred at around $0.31$ instead of the true distribution of $p$ which should be $\text{Unif}(0.25,0.4)$.

From what I was learning, it seems like Bayesian methods can have two interpretations – 1) where we use the prior to model the belief about the parameters but the parameters can be fixed and 2) where the parameters themselves follow some sort of a distribution. So why doesn't this work? Does this make sense theoretically? And does it have a link to hierarchical modeling as well?

Answer 1

In the Bayesian framework the prior is considered known and not something to be recovered. When you turn a prior into a posterior you condition on observed data. You correctly identified that a belief interpretation can be applied to both the prior and the posterior. This prior and posterior belief is in regards to the unknown fixed true $p$, before and after the data are observed.

When you selected $p$'s from $\text{Uniform}(0.25,0.4)$ and generated "observable" data, this represents a prior predictive distribution. It is not how the actual data generative process operates (because in truth $p$ is an unknown fixed constant), but the prior predictive distribution represents a prediction for a single as-of-yet unobserved $x$ based on not knowing the true $p$. It is predictive belief. Since the prior predictive distribution does not produce observable data, it is not something that would be used to construct a posterior.

If someone handed you a posterior and the observed data you could in principle reverse engineer the prior or something close to it, but the prior is considered known or given.