Questions tagged [posterior]

In Bayesian statistics, the term 'posterior' refers to the probability distribution of a parameter conditioned on the observed data.

314 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9 votes
0 answers
3k views

Implementing Predictive Posterior Distribution Using Stan

Background I had an example that sought to demonstrate the posterior predictive distribution in the context of a normal measurement model. The data that was used is as follows: ...
The Pointer's user avatar
  • 1,566
9 votes
0 answers
1k views

Density estimation/approximation from MCMC samples

I'm looking to accurately describe the density function of a multivariate posterior probability distribution based on samples from MCMC. As far as I know, in most cases this is done either with a ...
user3025882's user avatar
8 votes
0 answers
290 views

Time evolution of a Bayesian posterior

I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$, The data generating ...
Michael's user avatar
  • 3,278
8 votes
0 answers
550 views

Dealing with dependent data in a Bayesian model

Background: Consider a series of dependent data points, $$ y_1,y_2,y_3,\cdots,y_N. $$ In cases where the dependence is well described by an exponentially decaying auto-correlation function, it is ...
jonalm's user avatar
  • 131
8 votes
1 answer
137 views

What do the terms "nearly-optimal rate", "near-minimax rate", "minimax optimal rate" and "minimax rate" mean in the context of posterior consistency?

Definition: A sequence $\epsilon_n$ is a posterior contraction rate at the parameter $θ_0$ if $$\Pi_n(θ: d(θ, θ_0) ≥ M_n \epsilon_n| X^{(n)}) → 0$$ in $P^{(n)}_{θ_0}$-probability, for every $M_n → ∞$. ...
user3911153's user avatar
7 votes
0 answers
191 views

Including feature-dependent priors on output class, in bayesian logistic regression

When doing logistic regression with data $D_N = \{(x_i, y_i)\}_i^N$ with $x_i \in \mathbf{X}^N$ (each data point has N features) and $y_i \in \mathbf{Y}$ being assigned output classes, in a Bayesian ...
hirschme's user avatar
  • 1,110
7 votes
0 answers
2k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
Alex Crychek's user avatar
7 votes
0 answers
2k views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
user avatar
7 votes
1 answer
978 views

Generate Posterior predictive distribution at every step in the MCMC chain for a hierarchical regression model

I'm trying to fit a Bayesian Hierarchical regression model with a random correlated coefficients using R ,I'm using data having 160 groups (schools) to fit a model of math score as a function of one ...
Bahgat Nassour's user avatar
6 votes
1 answer
231 views

Lipschitzness of posterior mean of Gaussian process?

Let $T$ be a compact set, and let $K \colon T \times T \to \mathbb{R}$ be a positive definite kernel. Consider the canonical pseudo-distance $$d_K(x,y) = \sqrt{K(x,x) + K(y,y) - 2 K(x,y)}.$$ Let $f$ ...
ccriscitiello's user avatar
5 votes
0 answers
749 views

How do I sample from the posterior distribution with gamma likelihood with unknown alpha and beta?

I realize that this Wikipedia page provides the proportional form of the conjugate prior to the gamma distribution with unknown $\alpha$ and $\beta$ parameters, as well as the posterior values of $p$, ...
Brash Equilibrium's user avatar
4 votes
0 answers
2k views

Posterior distribution under Cauchy prior?

I have a (I hope) simple question! If I had a linear regression, $Y_t = \alpha + \beta X_t + \epsilon_t$ with $\epsilon_t \sim N(0,\sigma^2)$ and I assume a Cauchy prior for $\sigma$, is it ...
Chucky's user avatar
  • 153
4 votes
0 answers
1k views

Implementation of Bayes posterior predictive check

I have a question concerning the implementation of a bayes posterior predictive check. Let us assume i have this model (implementation is in R and jags): ...
IcePic's user avatar
  • 61
4 votes
0 answers
452 views

How to use an initial posterior for recursive / sequential updating in WinBUGS

I am using WinBUGS to estimate / update the parameters of a model. The model is: $$ \begin{aligned} D(T,B,a)&= B*(a_0+a_1T+a_2T^2+a_3T^3)+error(B,T,a) \\ error &= \mathcal N(0, B^{0.5}a_4(...
user21245's user avatar
4 votes
0 answers
1k views

Calibrating multiple binary SVM classifiers for one-vs-all multi-class classification

I'm classifying text using the one-vs-all approach. There are three classes. I've trained 3 different binary SVM classifiers using 10-fold cross-validation. The accuracy of the binary classifiers ...
mpenkov's user avatar
  • 141
3 votes
0 answers
222 views

Why do Posterior Sampling when we can Bootstrap?

I have just taken a stats course where half of the content was to do with sampling from posterior distributions. However, it was not clear why we were doing this when we could simply perform bootstrap ...
Cameron Chandler's user avatar
3 votes
0 answers
480 views

Posterior distribution of $\sigma^2$

In chapter 9 of Jim Albert's Bayesian computation with R it's mentioned that, in the context of Normal Linear Regression, the posterior joint density is: $$g(\beta, \sigma^2 | y) =g(\beta|y, \sigma^2)...
Maverick Meerkat's user avatar
3 votes
0 answers
1k views

How do I perform an actual "posterior predictive check"?

This question is the follow-up of this previous question: Bayesian inference and testable implications. For concreteness, consider the following bayesian model. This model is not to be taken ...
user avatar
3 votes
0 answers
373 views

What is the posterior kernel lengthscale of a Gaussian process?

If I have access to multiple samples from a Gaussian process with known covariance kernel but unknown parameters (i.e. unknown lengthscale), it is straightforward to estimate the lengthscale using ...
prdnr's user avatar
  • 91
3 votes
0 answers
251 views

Which optimizer use for laplace approximation

I have been trying to estimate the marginal posterior for D variable using Laplace approximation: $p(\theta_i) \approx \left[\frac{\det{H}}{2\pi\det{H(\theta_i)}}\right]^{1/2} \exp\left[-L(\theta_i, \...
Charlotte's user avatar
  • 538
3 votes
0 answers
66 views

Bayesian Decision Making (for particular problem)

I've read several papers why p-values should be replaced by Bayes factors and trying to use them. What I have: say, I have matrix of 2000 rows and 1000 columns. In each column I need to make a ...
German Demidov's user avatar
3 votes
0 answers
322 views

Multivariate posterior predictive distribution for the normal model (reference request)

Consider a Gaussian sample $y_1, \ldots, y_n \sim_{\text{iid}} \mathcal{N}(\mu,\sigma^2)$ and treat it in the Bayesian way with the noninformative prior $\pi(\mu, \sigma) \propto \frac{1}{\sigma}$. I ...
Stéphane Laurent's user avatar
3 votes
0 answers
45 views

Using Monte Carlo to find a posterior probability distirbution (distirbution propagation)

Using Monte Carlo to propagate error is a well known technique. To do that, one usually uses the Markov equation to find the posterior distribution $$ P(y|\mathbf{a})= \int \delta(y-F(\zeta))P(\zeta ...
Alexander Cska's user avatar
3 votes
0 answers
490 views

Gaussian processes: posterior vs. predictive distributions

I'm a little bit confused about the posterior distribution for Gaussian processes. If we consider the case with noise-free data, we have a prior $f|X \sim N(0, K)$. If we are given $\{(x_i, f_i): i=1.,...
theQman's user avatar
  • 647
3 votes
0 answers
460 views

What is popular choice for modelling distribution of orthogonal matrices

In Bayesian statistics, instead of considering a variable to be fixed and use MLE to infer that value, we put a prior distribution over that variable. Now consider an orthogonal matrix $W \in R^{d \...
Phong Le's user avatar
  • 341
3 votes
0 answers
140 views

Why do Srivastava et al. claim that "the best" theoretical regularization technique involves all possible network parameter settings?

In the original paper on Dropout by Srivastava, Hinton, Krizhevsky et al. (2014), the authors make this claim in the introduction: With unlimited computation, the best way to "regularize" a fixed-...
kdbanman's user avatar
  • 797
3 votes
0 answers
3k views

How to find MLE and MAP of a Poisson distribution?

Can someone explain how to find out the Maximum Likelihood and Maximum A Posteriori Estimate for a Poisson distribution with mean $\lambda$ ? I did the calculation for the MLE as follows: The ...
Rahul Bohare's user avatar
3 votes
0 answers
72 views

How should three unordered categories be encoded in a bayesian network framework?

The SAS FAQ suggest that for unordered two categories I should one dummy variables, for example: The common practice of using target values of .1 and .9 instead of 0 and 1 prevents the outputs of ...
kenorb's user avatar
  • 559
3 votes
0 answers
191 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
lacerbi's user avatar
  • 5,166
3 votes
0 answers
51 views

Derive ditribution for $\mu | Y_1,...Y_h,\rho $ (Bayesian stats)

I am trying to understand the following paper (http://www.ncbi.nlm.nih.gov/pubmed/20156954). Imagine we have H clinical trials with historical data on control group. $ Y_1, ... Y_h $ - are estimates ...
Ana's user avatar
  • 31
3 votes
0 answers
215 views

Derive Marginal Posterior to set up Gibbs-Sampler

I am currently trying to replicate a Hierarchical Model for multivariate returns proposed in the paper Portfolio selection using hierarchical Bayesian analysis and MCMC methods. However, in order to ...
Stefan Voigt's user avatar
  • 1,330
3 votes
0 answers
194 views

Does a proper prior lead always a proper posterior?

Does a proper prior lead always a proper posterior? I cannot check whether the posterior is proper, so I was wondering if this assumption is always satisfied .
sciury's user avatar
  • 41
3 votes
0 answers
338 views

Finding the Complicated Posterior Probability Distribution of $θ$

Suppose, we are given a likelihood function, $f(x|θ)$ corresponding to a shifted-exponential distribution and the prior distribution on the parameter $θ$ is a standard Cauchy distribution. Now I am ...
Dwaipayan Gupta's user avatar
3 votes
0 answers
95 views

Is the posterior a sufficient statistic when observations are conditionally independent?

Suppose there are two random variables, $X_1$ and $X_2$, and we're trying to infer $\theta$. If $X_1$ and $X_2$ are conditionally independent, then is $f(\theta|X_1)$ a sufficient statistic for $X_1$?...
Tom Cunningham's user avatar
3 votes
0 answers
207 views

Marginal Likelihood Latent Variable Model

I am trying to apply the method proposed by Chib in Marginal Likelihood from the Metropolis Hastings Output to calculate the marginal likelihood of a logit model the includes latent variables. ...
Ida's user avatar
  • 31
3 votes
0 answers
515 views

Best method to estimate the mean of a normal distribution?

Let $X = ( x_1, ..., x_n ) $ be $n$ samples from a normal distribution with unknown mean. What is the best estimator for this mean? I can think of at least 2 unbiased estimators: The empirical mean $...
usual me's user avatar
  • 1,227
3 votes
0 answers
306 views

Determining the posterior distribution for an Autoregressive or order 1 model

Question: For this question, note that the notation $y_{1:T} = (y_1, y_2, \cdots, y_T)$, ie, a vector of random variables. Consider the following AR(1) model: \begin{align*} y_{t+1} = \phi y_t + \...
TeTs's user avatar
  • 889
3 votes
0 answers
121 views

Practical problems with difficult posteriors

I'm looking for difficult Bayesian inference problems to test out different Monte Carlo sampling methods. I've mostly been looking at Hamiltonian Monte Carlo based algorithms and in particular, I've ...
John Salvatier's user avatar
2 votes
1 answer
9k views

Find posterior distribution for uniform distribution

Given X with uniform distribution in the interval [μ,μ+θ]. Suppose θ is given. Find the posterior distribution with prior distribution on your own. From that, find the Bayesian estimator with ...
Sang Huynh's user avatar
2 votes
0 answers
24 views

How do I evaluate correlation of model parameters using MCMC posterior samples from a rstan fit?

Is there a better way to do so than simply by taking posterior parameter estimates and calculating the Spearman or Pearson correlation between them? Anything specific to having posterior samples from ...
thlpswm's user avatar
  • 21
2 votes
1 answer
80 views

Regarding the bayes rule derivation of posterior distribution, $p(\omega|x,y),$ for a given dataset $D$ over $\omega.$

So I was going through this paper and under Uncertainty modeling it says So I tried deriving it on my own and I got $p(\omega | X, Y) = \frac{p(Y | X, \omega) \cdot p(X,\omega)}{p(Y | X) \cdot P(X)}$ ...
mutli-arm-bandit's user avatar
2 votes
0 answers
28 views

Distribution families whose likelihoods integrate to $+\infty$ for some sample values

I've recently started learning about Bayesian statistics, and I came across this very nice answer by Xi'an https://stats.stackexchange.com/a/129908/268693, which [in my slight paraphrasing] says the ...
Leonidas's user avatar
  • 121
2 votes
0 answers
170 views

Posterior predictive distribution for Bernoulli (and categorical)

I'm trying to confirm something I've tried to figure out about the posterior predictive distribution for Bernoulli vs. Binomial (and categorical vs. multinomial) random variables after a Bayesian ...
Björn's user avatar
  • 30.6k
2 votes
0 answers
99 views

Bound on the expectation of a function of random variable having a strictly log-concave probability density

let $\theta \in \mathbb{R}^d$ be a random variable having a strictly log-concave probability density function, i.e \begin{equation} p(\theta) = e^{-\phi(\theta)} \end{equation} where $\phi(\theta)$ is ...
zsheeba's user avatar
  • 21
2 votes
1 answer
76 views

How to obtain the distributions of linear model parameters?

For a linear model $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + …… + \sigma^2$$ Assuming $\beta_0, \beta_1,……$ are random varibles, how to obtain the distributions of these parameters? It seems that ...
Alex's user avatar
  • 83
2 votes
0 answers
164 views

When does this prior dominate likelihood?

This is a simple Bayesian inference problem, where we are trying to infer some weight parameter $w$. Our posterior distribution is $$ P\propto \exp\left(-\frac{1}{\sigma^2} w^Tw\right) \exp\left(-f(w)\...
CWC's user avatar
  • 281
2 votes
0 answers
233 views

Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition

I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
ad018's user avatar
  • 31
2 votes
0 answers
81 views

Show that posterior distribution is proportional to likelihood times prior when both y and X are in the equation

Reviewing MCMC for my work, I have got a problem with the very fundamental equation for the posterior: $$ P(\theta |y, X) = \frac{P(y|X, \theta)P(\theta)}{p(y|X)} = \frac{P(y|X, \theta)P(\theta)}{Z} $...
YoungMin Park's user avatar
2 votes
0 answers
70 views

Do most scientific discoveries commit the conditional probability fallacy?

Usually, in an experimental scientific paper where we have observations $X$ taken in laboratory conditions, we either reject some model $\mathcal{M}$ under the basis that $P(X \mid \mathcal{M}) \ll 1$,...
Bridgeburners's user avatar
2 votes
0 answers
187 views

Multivariate bayesian parameter estimation

I am implementing an example for pymc3 in python and I want to understand the mathematical formulation of this code. ...
MachineLearner's user avatar

1
2 3 4 5
7