In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior ...

learn more… | top users | synonyms

1
vote
1answer
19 views

Empirical Bayes vs “non-informative” priors

I am familiar with the mechanics with both methods, but don't know what factors I should consider when choosing between these two approaches for adjusting a prior. I would imagine that, on a case by ...
4
votes
2answers
55 views

How does one interpret the distribution over parameters in bayesian estimation?

I am new to Bayesian estimation. The assumption that the parameters are random variables seems a little unsettling to me. For example when considering a model for data, what physical interpretation ...
5
votes
2answers
80 views

What is the mathematical difference between using a un-informative prior and a frequentist approach?

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.) However, it seems to me that it would just make sense to use a frequentist approach instead. Why ...
3
votes
2answers
112 views

MCMC packages in R

Is there an R package for MCMC that can accept my self-defined (log)likelihood function (can be done in MCMCpack) and lets the user define contraints to the ...
3
votes
1answer
87 views

Deriving the posterior density for a lognormal likelihood and Jeffreys's prior

The likelihood function of a lognormal distribution is: $f(x; \mu, \sigma) \propto \prod_{i_1}^n \frac{1}{\sigma x_i} \exp \left ( - \frac{(\ln{x_i} - \mu)^2}{2 \sigma^2} \right ) $ and Jeffreys's ...
0
votes
1answer
41 views

Specifying a Laplace prior using a Gaussian random variable with Gamma variance

I need to place a Laplace prior on a random variable, however, I want to use a Gaussian distribution whose variance is Gamma(1,1) distributed, i.e., \begin{align} x &\sim N(\mu,\sigma^2)\\ ...
2
votes
0answers
40 views

Finding the most “uniform” or “least concentrated” density function, subject to moment constraints

Background I want to find a probability measure for a continuous random variable, subject to moment constraints, that is maximally "uniform", as defined below: Definition: Maximally Uniform ...
1
vote
1answer
75 views

In Bayesian hypothesis testing, do the prior model probabilities have to be equal?

The posterior odds is the product of the Bayes factor and prior odds: $\frac{p(M_1|data)}{p(M_2|data)}=\frac{p(data|M_1)}{p(data|M_2)}\times\frac{p(M_1)}{p(M_2)}$. I was under the impression that ...
2
votes
4answers
143 views

Is there a desription in the literature of a Normal hierarchical model with hyperparameters for both the mean and the standard deviation?

I'm looking for a comprehensive description of and justification for a Normal hierarchical model where both the means of the groups and the standard deviation are modelled. It is common to find ...
0
votes
1answer
45 views

Normal prior for Binomial likelihood [closed]

Pardon my ignorance, i am new to Bayesian Analysis. I am trying to use Normal prior for a binomial likelihood, which of these are most likely candidates ( $\bar{x} $, $ \mu $, $ \sigma $ ) ...
1
vote
1answer
89 views

Prior on a non identifiable parameter-MCMC integration

To introduce the problem I will explain the Projected normal distribution. Let $\mathbf{z}_i=(z_{i1},z_{i2})$ be a bivariate vector distributed as a bivariate normal with vector mean ...
1
vote
0answers
11 views

Setting up this generative model for inference: uniform priors

I am trying to set up a generative model where I have two images $x$ and $y$ and it is assumed that $y$ can be generated by applying some unknown transformation to $x$ i.e. $$ y = t(x, w) + e $$ ...
3
votes
0answers
37 views

Priors for discriminative methods?

Say we want to build a classifier for a binary classification problem using a discriminative method (e.g. SVM) and be able to impose a prior on the classes. For example, let's assume that we want to ...
1
vote
0answers
28 views

Conjugate Prior for Probit likelihood function

I am trying to do a Bayesian analysis in which my likelihood function is a probit function on two parameters. From various sources, I found out that Normal distribution is a conjugate prior to probit ...
1
vote
1answer
28 views

How to model the prior distribution of several Gaussians with known parameters

I might be wrong, I just feel that the following case is different from the problem of modelling observations with a conjugate prior: Suppose I have $n$ different Gaussians each with a different (but ...
2
votes
0answers
35 views

Haar prior for von Mises distribution

Ok, Let me tell you that this is the very first time that I have no idea with the question below. I can not find a solution or anything that will lead me to it. I say this to prevent comments "what ...
0
votes
1answer
30 views

Weakly informative priors for “r.v. $\mathbf{x}$ is not close to $\mathbf{x}_0$”?

Given some fixed vector $\mathbf{x}_0\in\mathbb{R}^d$, I want to put a prior on a random variable $\mathbf{x}\in\mathbb{R}^d$ so that "it's not very close to $\mathbf{x}_0$". For the moment I put a ...
2
votes
0answers
41 views

Defining constraint on prior with potential class

I have written an MCMC code in order to estimate parameters Xpos, Ypos, MASS and concentration with a set of input data gal_pos, ...
3
votes
2answers
159 views

What does it mean to integrate over the posterior?

I have been reading a book that cites an example where a uniform distribution is the initial prior, and then a person scores 9/10 on a test. Then the resulting posterior becomes the prior ...
2
votes
1answer
61 views

How to construct a reasonable prior and likelihood for Bayes modelling?

To apply Bayes inference for data analysis or machine learning, we have to construct prior and likelihood, right? But if I fail to come up with a reasonable prior and likelihood, then the Bayes model ...
0
votes
0answers
38 views

learning hyper parameters: are we allowed to touch the prior parameters after observing the data?

There are many algorithms/applications that aim to learn the hyper parameters i.e. the parameters of a prior distribution from the observed data. A typical algorithm works in an iterative function ...
1
vote
2answers
47 views

Use of the Jeffreys prior in multidimensional models

Suppose a model, $$x_{i} \sim N(\theta_{i}, \phi), \text{ for } i=1,\ldots,n$$ Furthermore, suppose the variance parameter, $\phi$, is some known constant. The multidimensional Jeffreys prior is ...
1
vote
0answers
37 views

Gibbs sampling with Log-Normal observations

I am writing a Gibbs sampler for data that is Log-Normal (LN) distributed, with unknown mean and variance. There is a wealth of information on inference for LN models when either the mean or variance ...
0
votes
0answers
20 views

Prior over transition matrices of markov chain

I want to be able to marginalize over the transition matrix of a markov model. The goal is to get the marginal likelihood on the number of states necessary to explain the data. Something that would ...
5
votes
3answers
374 views

How can an improper prior lead to a proper posterior distribution?

We know that in the case of a proper prior distribution, $P(\theta \mid X) = \dfrac{P(X \mid \theta)P(\theta)}{P(X)}$ $ \propto P(X \mid \theta)P(\theta)$. The usual justification for this step is ...
0
votes
2answers
49 views

Are discrete single value prior distributions always lost in MAP estimation?

I’d like to illustrate my problem with a little (heavily abbreviated) excercise. I think it will help a lot to stress my point. Meet Mary, Tom and Jane. They all are programmers. Mary is a decent ...
0
votes
1answer
51 views

Non-binomial posteriors for a binomial prior?

Let's assume we have a discrete binary random variable K (K=0 or K=1) for which the prior distribution is binomial. My understanding of Bayesian statistics tells me that regardless of the likelihood, ...
1
vote
1answer
35 views

How should I construct a prior distribution with a particular kind of count data

For context I will first explain the overall problem that I am working on. I am given a catalog of product names and I am also given a large text dataset that may contain mentions of these catalog ...
7
votes
2answers
294 views

You observe k heads out of n tosses. Is the coin fair?

I was asked this question with $(n, k) = (400, 220)$ in an interview. Is there a "correct" answer? Assume the tosses are i.i.d. and the probability of heads is $p=0.5$. The distribution of the ...
4
votes
2answers
311 views

Ridge regression – Bayesian interpretation

I have heard that ridge regression can be derived as the mean of a posterior distribution, if the prior is adequately chosen. Is the intuition that the constraints as set on the regression ...
0
votes
1answer
44 views

Distribution of a Mean and Variance

Say we have observations $x_1 \dots x_n$ and we have some sort of Bayesian framework where we would like to estimate a distribution for the mean $\mu$ of our observations and the variance $\sigma^2$ ...
2
votes
0answers
172 views

Hyper-prior for negative binomial in hierarchical model using JAGS/BUGS

Below I'm using a negative binomial because it is more flexible than a simple poisson model. The data are counts $y$ of events for 16 individuals $x$. There are 14 counts (i.e. counting periods) for ...
3
votes
1answer
51 views

Univariate priors for the parameters of a Beta distribution

I need a rather a prior on the parameters of a Beta distribution (i.e. $\alpha$ and $\beta$). I have an external constraint that requires me to use univariate priors, one for $\alpha$ and one ...
0
votes
0answers
11 views

Applying Beta distribution for calculation latent variables

I would like to find the probability distribution function for the below scenario,its similar to Computer Adaptive technique (IRT) I need to estimate the ability of a student from answered questions ...
2
votes
0answers
67 views

Non-informative prior for Poisson/gamma density

In the Albert book on Bayesian computation with R, exercise 4.8.5 (p.83), it is suggested to use $$ p(a, b) \sim (a \times b)^{-2} $$ as the non-informative prior for the Poisson/Gamma model: $$ ...
0
votes
0answers
26 views

Calculating posterior and prior odds

Question: Now, I'm confused about assigning probabilities here. I find $P(A^c|E) = (.001)(.99) = .00099$ and $P(E|A) = .99$, but what about the first two sentences? Does that mean that $P(E) = .001$ ...
4
votes
0answers
83 views

Can likelihood be changed when the prior changes?

I have a data which follows gamma distribution and want to know the uncertainty of the parameters of this data. $\text{Data} \sim \text{Gamma} (\alpha, \beta)$ Parameters $\alpha \sim \text{Gamma} ...
0
votes
0answers
25 views

Marginal Likelihood Prior

I have a model with probability matrix for a distribution of $x$, $y=\{0,1\}$, $p(x,y|w)$ where $w=[w_1,w_2,w_3,w_4]$ $p(x=0,y=0)=w_1$, $p(x=0,y=1)=w_2$ $p(x=1,y=0)=w_3$, ...
3
votes
1answer
55 views

How can a uniform prior make the posterior mean different from the MLE?

I read the following in Machine Learning: A Probabilistic Perspective: How can a uniform prior move the posterior mean? Isn't a uniform distribution supposed to not bias the result? Are there any ...
4
votes
1answer
87 views

Log odds as prior

I have this problem: In betting situations one is often interested in odds, referring in the thumbtack tossing $\theta / (1 - \theta)$. Alternatively one may consider the log-odds: ...
2
votes
2answers
104 views

Prior Gamma distribution: Select appropriate alpha given beta and median

I am trying to programatically select a prior distribution from the Gamma family of distributions. The primary criteria that I need to satisfy is that the median of the distribution should be a given ...
1
vote
1answer
39 views

Prior predictive with discrete prior

I'm working with a uniform distribution as a prior, defined as: $\pi(\theta) = \begin{cases} \frac{1}{7} & \text{if } \theta\in\{0,\frac{1}{6},\frac{2}{6},\ldots,1\} \\ 0 & ...
2
votes
1answer
85 views

Probabilistic modelling MCMC question with pyMC

This is my first post and I am a newby in pymc. I am trying to model a non-linear system (see below for a further explanation). I create my synthetic data with: ...
0
votes
1answer
63 views

Explicit prior versus implicit prior

I am reading a paper where they talk about keeping a prior explicit as opposed to an implicit prior. To be honest, I have never came across the terms explicit/implicit in context of priors and I was ...
1
vote
0answers
48 views

Figuring out quantiles in quantile regression

Suppose I have a dataset $\{y_i,x_i\}$ $i=1,2,...n$. For the response variable, $y_i$ as per quantile regression I have the following likelihood: $$p(y_i|\beta,\alpha_i,\sigma) ...
1
vote
2answers
104 views

what does “a distribution over distributions” mean?

I am reading a pdf about Dirichlet Process, and it said "A Dirichlet Process is also a distribution over distributions." anyone could explain this in plain English what does it mean by that? thanks ...
6
votes
1answer
146 views

Why not use Beta(1,1) as boundary avoiding prior on a transformed correlation parameter?

In Bayesian Data Analysis, chapter 13, page 317, second full paragraph, in the modal and distributional approximations, Gelman et al. write: If the plan is to summarize inference by the posterior ...
8
votes
3answers
221 views

Posterior very different to prior and likelihood

If the prior and the likelihood are very different from each other, then sometimes a situation occurs where the posterior is similar to neither of them. See for example this picture, which uses normal ...
1
vote
1answer
83 views

Specification of logical node (with distribution?) in WINBUGS

For a piece of homework I have an assignment using WINBUGS which I must admit confuses me to say the least. Tangential to my question but I have a few stochastic nodes that are to be gamma ...
4
votes
3answers
264 views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...