15

This is a good question. I'm going to use a simple example to illustrate my approach. Suppose I am working with someone who needs to provide me priors on the mean and the variance for a gaussian likelihood. Something like $$ y \sim \mathcal{N}(\mu, \sigma^2) $$ The question is: "What are this person's priors on $\mu$ and $\sigma^2$?" For the mean I ...


5

...example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator... There is a fundamental issue with this property, namely that it depends on the parameterisation of the sampling model. Indeed, if $$\hat\theta^\text{MAP}=\arg\max_\theta L(\theta|x)\pi(\theta)=\arg\max_\theta L(\theta|x)=\hat\theta^\text{MLE}\tag{1}$...


5

In the Bayesian formula: $$\text{posterior} = \,\frac{\text{likelihood} \cdot \text{prior}}{\text{normalizing constant}}$$ If we call the observations $y$ and the parameters $\theta$, then this equates: $$p(\theta | y) = \, \frac{p(y | \theta) \cdot p(\theta)}{p(y)}$$ Here, the normalizing constant $p(y)$ is calculated as: $$p(y) = \int p(y | \theta) \...


4

The choice of a prior distribution is based on prior belief, prior information, or some constructive principle, like minimum information, maximum entropy, frequentist matching, leading to "default" or "reference" (rather than "noninformative") priors. However, there is no unique and no better/best choice for a prior as the Bayesian perspective is conditional ...


4

The prior which is uniform over $[m,\infty)$ is improper, but informative: it contains the information that the value is at least $m$.


3

This is an example of where a Bayesian estimator is examined with respect to its frequentist properties ---i.e., how it behaves over repeated trials conditional on the parameter value. Although Bayesian estimators are derived using a prior distribution over the possible parameter values, ultimately, the estimators are functions of the data (and possibly ...


2

Yes, it's possible, since you can write M-estimation in terms of a loss function (the rho function), to which you can add a penalty, reducing it to another optimization problem. However some M-estimators can have multiple modes on the likelihood, which L1 or L2 regularization won't necessarily remove. While M-estimation arises from likelihood ideas, it doesn'...


1

I would look into the horseshoe-type markov random fields (MRFs) described in https://projecteuclid.org/euclid.ba/1487905413. They generalize Gaussian MRFs to include horseshoe-type priors which induce the type of shrinkage of the coefficients you want, while maintaining the autoregressive properties from a random walk. They also have a Stan implementation!


Only top voted, non community-wiki answers of a minimum length are eligible