Questions tagged [posterior]

Refers to the probability distribution of parameters conditioned on data in Bayesian statistics.

250 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
356 views

Dealing with dependent data in a Bayesian model

Background: Consider a series of dependent data points, $$ y_1,y_2,y_3,\cdots,y_N. $$ In cases where the dependence is well described by an exponentially decaying auto-correlation function, it is ...
6
votes
0answers
154 views

Including feature-dependent priors on output class, in bayesian logistic regression

When doing logistic regression with data $D_N = \{(x_i, y_i)\}_i^N$ with $x_i \in \mathbf{X}^N$ (each data point has N features) and $y_i \in \mathbf{Y}$ being assigned output classes, in a Bayesian ...
6
votes
0answers
2k views

Implementing Predictive Posterior Distribution Using Stan

Background I had an example that sought to demonstrate the posterior predictive distribution in the context of a normal measurement model. The data that was used is as follows: ...
6
votes
0answers
183 views

Time evolution of a Bayesian posterior

I have a question regarding the time evolution of a quantity related to a Bayesian posterior. Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$, The data generating ...
6
votes
0answers
1k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
6
votes
0answers
2k views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
6
votes
1answer
686 views

Generate Posterior predictive distribution at every step in the MCMC chain for a hierarchical regression model

I'm trying to fit a Bayesian Hierarchical regression model with a random correlated coefficients using R ,I'm using data having 160 groups (schools) to fit a model of math score as a function of one ...
5
votes
0answers
531 views

Density estimation/approximation from MCMC samples

I'm looking to accurately describe the density function of a multivariate posterior probability distribution based on samples from MCMC. As far as I know, in most cases this is done either with a ...
5
votes
0answers
656 views

How do I sample from the posterior distribution with gamma likelihood with unknown alpha and beta?

I realize that this Wikipedia page provides the proportional form of the conjugate prior to the gamma distribution with unknown $\alpha$ and $\beta$ parameters, as well as the posterior values of $p$, ...
4
votes
0answers
1k views

Posterior distribution under Cauchy prior?

I have a (I hope) simple question! If I had a linear regression, $Y_t = \alpha + \beta X_t + \epsilon_t$ with $\epsilon_t \sim N(0,\sigma^2)$ and I assume a Cauchy prior for $\sigma$, is it ...
4
votes
0answers
424 views

How to use an initial posterior for recursive / sequential updating in WinBUGS

I am using WinBUGS to estimate / update the parameters of a model. The model is: $$ \begin{aligned} D(T,B,a)&= B*(a_0+a_1T+a_2T^2+a_3T^3)+error(B,T,a) \\ error &= \mathcal N(0, B^{0.5}a_4(...
4
votes
0answers
961 views

Calibrating multiple binary SVM classifiers for one-vs-all multi-class classification

I'm classifying text using the one-vs-all approach. There are three classes. I've trained 3 different binary SVM classifiers using 10-fold cross-validation. The accuracy of the binary classifiers ...
3
votes
1answer
80 views

Finding the posterior mean

I have been trying to solve the following problem: Suppose $X_1,...,X_n$ are iid exponential random variables, with density $f(x;\theta) =\theta e^{-\theta x}$ ,and let us suppose that we have a prior ...
3
votes
0answers
235 views

How do I perform an actual “posterior predictive check”?

This question is the follow-up of this previous question: Bayesian inference and testable implications. For concreteness, consider the following bayesian model. This model is not to be taken ...
3
votes
0answers
40 views

Reproducing a didactic example of Lindley (1993)

Lindley (1993) discusses the following mixed discrete and continuous prior for the tea tasting lady experiment, where $\pi$ is probability of a correct classification: $p(\pi=0.5) = 0.8$ (discrete ...
3
votes
0answers
192 views

What is the posterior kernel lengthscale of a Gaussian process?

If I have access to multiple samples from a Gaussian process with known covariance kernel but unknown parameters (i.e. unknown lengthscale), it is straightforward to estimate the lengthscale using ...
3
votes
0answers
108 views

Which optimizer use for laplace approximation

I have been trying to estimate the marginal posterior for D variable using Laplace approximation: $p(\theta_i) \approx \left[\frac{\det{H}}{2\pi\det{H(\theta_i)}}\right]^{1/2} \exp\left[-L(\theta_i, \...
3
votes
0answers
61 views

Bayesian Decision Making (for particular problem)

I've read several papers why p-values should be replaced by Bayes factors and trying to use them. What I have: say, I have matrix of 2000 rows and 1000 columns. In each column I need to make a ...
3
votes
1answer
198 views

How to predict using Spatial temporal hierarchical bayesian models

I am using the R package CARBayesST to fit a Spatial-temporal Bayesian models. I want to use piece-wise ST model proposed by Lee and Lawson, 2017. The package does not have a built-in predict ...
3
votes
0answers
247 views

Multivariate posterior predictive distribution for the normal model (reference request)

Consider a Gaussian sample $y_1, \ldots, y_n \sim_{\text{iid}} \mathcal{N}(\mu,\sigma^2)$ and treat it in the Bayesian way with the noninformative prior $\pi(\mu, \sigma) \propto \frac{1}{\sigma}$. I ...
3
votes
0answers
36 views

Using Monte Carlo to find a posterior probability distirbution (distirbution propagation)

Using Monte Carlo to propagate error is a well known technique. To do that, one usually uses the Markov equation to find the posterior distribution $$ P(y|\mathbf{a})= \int \delta(y-F(\zeta))P(\zeta ...
3
votes
0answers
173 views

What is popular choice for modelling distribution of orthogonal matrices

In Bayesian statistics, instead of considering a variable to be fixed and use MLE to infer that value, we put a prior distribution over that variable. Now consider an orthogonal matrix $W \in R^{d \...
3
votes
0answers
131 views

Why do Srivastava et al. claim that “the best” theoretical regularization technique involves all possible network parameter settings?

In the original paper on Dropout by Srivastava, Hinton, Krizhevsky et al. (2014), the authors make this claim in the introduction: With unlimited computation, the best way to "regularize" a fixed-...
3
votes
0answers
2k views

How to find MLE and MAP of a Poisson distribution?

Can someone explain how to find out the Maximum Likelihood and Maximum A Posteriori Estimate for a Poisson distribution with mean $\lambda$ ? I did the calculation for the MLE as follows: The ...
3
votes
0answers
67 views

How should three unordered categories be encoded in a bayesian network framework?

The SAS FAQ suggest that for unordered two categories I should one dummy variables, for example: The common practice of using target values of .1 and .9 instead of 0 and 1 prevents the outputs of ...
3
votes
0answers
155 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
3
votes
0answers
45 views

Derive ditribution for $\mu | Y_1,…Y_h,\rho $ (Bayesian stats)

I am trying to understand the following paper (http://www.ncbi.nlm.nih.gov/pubmed/20156954). Imagine we have H clinical trials with historical data on control group. $ Y_1, ... Y_h $ - are estimates ...
3
votes
0answers
187 views

Derive Marginal Posterior to set up Gibbs-Sampler

I am currently trying to replicate a Hierarchical Model for multivariate returns proposed in the paper Portfolio selection using hierarchical Bayesian analysis and MCMC methods. However, in order to ...
3
votes
0answers
171 views

Does a proper prior lead always a proper posterior?

Does a proper prior lead always a proper posterior? I cannot check whether the posterior is proper, so I was wondering if this assumption is always satisfied .
3
votes
0answers
252 views

Finding the Complicated Posterior Probability Distribution of $θ$

Suppose, we are given a likelihood function, $f(x|θ)$ corresponding to a shifted-exponential distribution and the prior distribution on the parameter $θ$ is a standard Cauchy distribution. Now I am ...
3
votes
0answers
70 views

Is the posterior a sufficient statistic when observations are conditionally independent?

Suppose there are two random variables, $X_1$ and $X_2$, and we're trying to infer $\theta$. If $X_1$ and $X_2$ are conditionally independent, then is $f(\theta|X_1)$ a sufficient statistic for $X_1$?...
3
votes
0answers
968 views

Implementation of Bayes posterior predictive check

I have a question concerning the implementation of a bayes posterior predictive check. Let us assume i have this model (implementation is in R and jags): ...
3
votes
0answers
156 views

Marginal Likelihood Latent Variable Model

I am trying to apply the method proposed by Chib in Marginal Likelihood from the Metropolis Hastings Output to calculate the marginal likelihood of a logit model the includes latent variables. ...
3
votes
0answers
230 views

Best method to estimate the mean of a normal distribution?

Let $X = ( x_1, ..., x_n ) $ be $n$ samples from a normal distribution with unknown mean. What is the best estimator for this mean? I can think of at least 2 unbiased estimators: The empirical mean $...
3
votes
0answers
272 views

Determining the posterior distribution for an Autoregressive or order 1 model

Question: For this question, note that the notation $y_{1:T} = (y_1, y_2, \cdots, y_T)$, ie, a vector of random variables. Consider the following AR(1) model: \begin{align*} y_{t+1} = \phi y_t + \...
3
votes
0answers
117 views

Practical problems with difficult posteriors

I'm looking for difficult Bayesian inference problems to test out different Monte Carlo sampling methods. I've mostly been looking at Hamiltonian Monte Carlo based algorithms and in particular, I've ...
2
votes
1answer
4k views

Find posterior distribution for uniform distribution

Given X with uniform distribution in the interval [μ,μ+θ]. Suppose θ is given. Find the posterior distribution with prior distribution on your own. From that, find the Bayesian estimator with ...
2
votes
0answers
33 views

Posterior distribution of $\sigma^2$

In chapter 9 of Jim Albert's Bayesian computation with R it's mentioned that, in the context of Normal Linear Regression, the posterior joint density is: $$g(\beta, \sigma^2 | y) =g(\beta|y, \sigma^2)...
2
votes
2answers
47 views

Laplace approximation in high-dimensions

Obviously computing the inverse Hessian is hard when a probability distribution is fitted on high-dimensional datapoints. One idea to reduce computational cost would be to approximate the distribution ...
2
votes
0answers
35 views

Single point estimate of joint posterior distribution

This is a very basic Bayesian beginner question about what to do when one does not want a full parameter distribution but only a single set of parameters. Example: Let's say we have a number of points ...
2
votes
0answers
61 views

How to determine the size of highest density region in high dimensions

I want to calculate the "size" of the highest density region (HDR) that contains p% of the total probability for multivariate samples of a Bayesian posterior obtained via MCMC. In 1D this "size" is ...
2
votes
0answers
21 views

Bayesian inference problem with three parameters

Assume that I observe a yearly time series for the number of certain events occurring per year. Also, assume that the data points come from a Poisson process with parameter $\lambda$. The dataset in ...
2
votes
0answers
50 views

Posterior distribution of Bernoulli distribution

The pdf of X | $\theta$ is given by $\theta^x (1- \theta)^{1-x}$ and its prior distribution is given by $p(\theta) \frac {1} {B(\alpha, \beta)} \theta^{\alpha - 1} (1 - \theta)^{\beta - 1}$ where $...
2
votes
0answers
46 views

Threshold crossing: Integrating posterior distribution over a forecast period

Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model: $$p(Y_{t+h} \mid Y_{1:t}, \theta)$$ for any horizon $h$, and some parameters $\theta$. Now ...
2
votes
1answer
52 views

Approximating p-values from Bayesian posterior distribution

I'm preparing a manuscript for publication in which I have fit a mixed linear model using Bayesian regression. I'm assessing whether group A is bigger than group B. In the paper, I have reported the ...
2
votes
0answers
36 views

Posterior with a much larger uncertainty than the prior

I have done an MCMC analysis with many variables. One of my nuisance parameters has a Normal prior distribution with mean 0 and standard deviation 1. The posterior distribution for this parameter has ...
2
votes
0answers
59 views

Multivariate bayesian inference: learnig about the mean of a variable by observing another variable

I want to derive a Bayesian learning procedure where I don't only learn from my own signal, but also from other signals which are correlated to mine. I thought it could simply work with Bayesian ...
2
votes
0answers
52 views

Finding the mode of the posterior distribution

I have the following hierachical bayesian model - $\mathbf{x}|\mathbf{c},\sigma^2 \sim \mathcal{N}(\mathbf{x}|\mathbf{c},\sigma^2)$ $\mathbf{c}|\mathbf{c}_1,\sigma^2_2 \sim \mathcal{N}(\mathbf{c}|\...
2
votes
0answers
26 views

Model selection for this model with one observation

I would like to perform model selection given a range of $k$ models $\mathcal{M}_1, \mathcal{M}_2, ..., \mathcal{M}_k$, each with some prior probability $f(\mathcal{M}_1), \dots, f(\mathcal{M}_k).$ ...
2
votes
0answers
266 views

Posterior sampling without using pm.Potential in pyMC3

I'm going through the Price Is Right example in chapter 5 of Probabilistic Programming & Bayesian Methods for Hackers and I have problems understanding the solution. I have tried to change the ...

1
2 3 4 5