Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [posterior]

Refers to the probability distribution of parameters conditioned on data in Bayesian statistics.

Filter by
Sorted by
Tagged with
1
vote
0answers
20 views

What do these equations on Bayesian regression (MAP) from Chapter 3.3 in PRML by Bishop mean?

This was taken from Ch 3.3 on Bayesian Linear Regression from Pattern Recognition in Machine Learning by Bishop. Apparently the posterior can be described by eq 3.49. Eq 3.48 represents the prior ...
1
vote
1answer
24 views

Explanation of Equation 5.3 from Gaussian Processes for Machine Learning

I am currently reading through C. E. Rasmussen & C. K. I. Williams' Gaussian Processes for Machine Learning and was going through chapter 5. I could not exactly understand the derivation of ...
0
votes
0answers
19 views

Find posterior distribution given prior distribution

Problem Be $X_1,...,X_n$ a random sample of $X$ ~ $Geometric(\theta)$, i.e., $f(x|\theta)=\theta(1-\theta)^x \forall x = 0,1,2,...$ Assuming a prior distribution for $\theta$ find the posterior ...
0
votes
0answers
32 views

Posterior Predictive Distribution for Uniform Likelihood and Pareto Prior

I'm trying to find the posterior predictive distribution for data $X_i, \dots X_n$ from a a $Uniform [0, \theta]$ distribution. The prior distribution for $\theta$ is a $Pareto[\alpha, \beta]$ ...
0
votes
0answers
9 views

Deriving full conditional of ordered probit model (Bayes)

I have a question regarding the following exercise: I am able to compute the complete (full) data likelihood function, the full conditionals of $y^{*}_i$ and $\beta$. However, I do not know how to ...
0
votes
0answers
15 views

Dynamically updating posterior density in R

I want to redefine my function in a loop by calling the function from last iteration. However I know this is basically a recursive way which I don't want. To give an example, see the following ...
1
vote
1answer
26 views

Range of integration for joint and conditional densities

Did I mess up the range of integration in my solution to the following problem ? Consider an experiment for which, conditioned on $\theta,$ the density of $X$ is \begin{align*} f_{\theta}(x) = \...
0
votes
0answers
18 views

Generating marginal posteriors from MCMC output in two-factor model

Quick summary: if I have a MCMC sample of the posterior distribution of two factors and their interactions, can I marginalize out one factor simply by concatenating the posterior samples from each ...
0
votes
0answers
11 views

Parameter estimation: Chi^2 vs. posterior density

Let's say I want to estimate the mean of a sample of i.i.d normal random variables. For this I use two methods: Compute Chi^2 as $\chi^2 = \sum (x_i - \mu)^2/\sigma^2$ for a range of possible means ...
1
vote
1answer
30 views

Bayesian Linear Regression, trouble with posterior. Variance equal identity

I am trying to solve the following problem. If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
3
votes
2answers
37 views

What is the conceptual difference between posterior and likelihood? [duplicate]

I have trouble discerning conceptually between these two notions. I am aware of their formal relations, proprieties and what not, but I just can't wrap my head around what they "mean", if that even ...
0
votes
0answers
28 views

$X\sim \mathcal{N}(\theta,\sigma^2)$, $\pi(\theta,\sigma^2)\propto 1/\sigma^2$, $Y\sim \mathcal{N}(\rho X,\sigma^2)$, $\rho$ fixed. $f(y|x)$?

like in the title I have the following question. Let $X\sim \mathcal{N}(\theta,\sigma^2)$ with the improper prior $\pi(\theta,\sigma^2)\propto 1/\sigma^2$ and consider $Y\sim \mathcal{N}(\rho X,\...
0
votes
1answer
111 views

GIBBS sampling : do samples for the subset of variables approxiamate the related marginal distribution?

I'm reading the page Gibbs sampling on wikipedia. I really don't understand why the following statement is true. "The marginal distribution of any subset of variables can be approximated by simply ...
1
vote
0answers
46 views

How to start coding for posterior inference

I am trying to implement the model given in http://proceedings.mlr.press/v84/andersen18a/andersen18a.pdf where they have used mean-field variational inference for posterior inference, but I want to ...
2
votes
0answers
24 views

Reproducing a didactic example of Lindley (1993)

Lindley (1993) discusses the following mixed discrete and continuous prior for the tea tasting lady experiment, where $\pi$ is probability of a correct classification: $p(\pi=0.5) = 0.8$ (discrete ...
0
votes
1answer
23 views

How does Thomson Sampling work in a real world application of Multi Armed Bandit Testing

I understand the basics of Thomson Sampling, but how is it implemented in practice? If there are three variants each with a 1/3 of traffic allocated to them on day 1, how is traffic dynamically ...
0
votes
0answers
21 views

Calculating probability from posterior distribution with continuous parameter space

Suppose that we are interested in the parameter $\theta$, and that our prior distribution for $\theta$ is given by $Beta(a=.1, b=.9)$. For the binary random variable $Y$ that takes value $1$ with ...
0
votes
1answer
38 views

Finding the posterior distribution given an improper prior

Let $X \sim N(\theta, \sigma^2)$ where $\sigma^2$ is known. Let the prior density $\pi(\theta) =1, \theta \in \mathbb{R}$ to be the improper uniform density over the real line. Find the posterior ...
0
votes
1answer
47 views

Questions about the principles of Bayesian analysis + R [closed]

Let's say I have a data of flywing lengths which is identically distributed (normal). (data: https://seattlecentral.edu/qelp/sets/057/s057.txt). I want to estimate the mean (theta). I have to choose ...
0
votes
0answers
10 views

Posterior probability and linear discriminant analysis

Is it possible, using any form of the posterior probability calculations, to assign a cutoff point in linear discriminant analysis based on a posterior probability value? Hypothetical scenario: We ...
2
votes
2answers
42 views

Borrowing observations for prior probability in Bayesian Inference

For the purposes of Bayesian Inference, is it assumed that the historical observations used for the prior probability values must be from the exact entity for which you are looking to calculate the ...
1
vote
1answer
32 views

Uniform Prior on Normal Mean with Known Variance Implies Truncated Normal Posterior?

Let's say I have a uniform prior $\mu \sim \mathcal{U}(a,b)$, a normal likelihood $y|\mu \sim \mathcal{N}(\mu,\sigma^2)$ with known variance $\sigma^2$, and one observation $y$. Is then the posterior $...
0
votes
1answer
20 views

Is there a derivation for the Posterior Predictive Distribution?

I came across this term in the deep learning book: $p(x_{m+1}|x_1 ... x_m) = \int p(x_{m+1}|\theta)p(\theta|x_1 ... x_m)d\theta$ After some research I find that this term is the definition of the ...
2
votes
0answers
25 views

Posterior distribution of Bernoulli distribution

The pdf of X | $\theta$ is given by $\theta^x (1- \theta)^{1-x}$ and its prior distribution is given by $p(\theta) \frac {1} {B(\alpha, \beta)} \theta^{\alpha - 1} (1 - \theta)^{\beta - 1}$ where $...
0
votes
0answers
26 views

Representativity of a sample to a reference population with a mixed distribution

I a have a series of small samples taken not randomly from a reference population with a complex distribution over a categoric variable with 21 levels and a continuous one in the x > 0 domain. I ...
0
votes
0answers
22 views

Reducing dimension $P(\theta|y) = P(\theta|s)$ in the posterior distribution

Given a sample of $n$ independent observations $\boldsymbol{y}$. Let $S(\boldsymbol{y})$ be a sufficient statistic for the underlying parameter $\boldsymbol{\theta}$ so that the density can be ...
1
vote
0answers
24 views

Threshold crossing: Integrating posterior distribution over a forecast period

Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model: $$p(Y_{t+h} \mid Y_{1:t}, \theta)$$ for any horizon $h$, and some parameters $\theta$. Now ...
0
votes
0answers
27 views

posterior predictive distribution for latent dirichlet allocation model

I want to obtain posterior predictive distribution on the LDA model, actually, I want to predict n next sample ( words in this model). can anyone help me? I attach the LDA model here, and it is ...
0
votes
1answer
19 views

Estimation of joint errors

Suppose we have multiple normal random variables, each which is a normal distribution with it's own mean but joint variance: $$X_i \sim N(\mu_i, \sigma^2)$$ Now suppose we collect data from these ...
0
votes
0answers
27 views

Nested sampling: estimate of bulk posterior support over prior

Going through the details of the Nested sampling Skilling paper, and I've encountered an estimate in Section 5 which I cannot reproduce. Rephrasing what's mentioned in the paper: we assume to have a ...
1
vote
0answers
22 views

Bayesian inference in independent but not identically distributed examples

Let data $\lbrace X_i \rbrace _{i=1}^m$ be examples sampled independently from normals, $X_i \sim N(\mu,\sigma_i)$. Variances are known but mean $\mu$ is unknown. Let prior on $\mu$ be also a Gaussian,...
2
votes
1answer
36 views

Approximating p-values from Bayesian posterior distribution

I'm preparing a manuscript for publication in which I have fit a mixed linear model using Bayesian regression. I'm assessing whether group A is bigger than group B. In the paper, I have reported the ...
0
votes
1answer
24 views

what is the step by step procedure of updating a posterior probability with new data coming in

I have a question about how to update posterior probability sequentially when new data comes in sequentially by say $x_1$, then $x_2$, then $x_3$,.... I understand this form when the first data $x_1$ ...
2
votes
0answers
31 views

Posterior with a much larger uncertainty than the prior

I have done an MCMC analysis with many variables. One of my nuisance parameters has a Normal prior distribution with mean 0 and standard deviation 1. The posterior distribution for this parameter has ...
0
votes
0answers
36 views

Expectation of exponential family distributions

Is there a closed form of the following marginal (one dimensional data) $\pi(\theta|y) = \mathbb{E}_{x \sim \pi_R(x|y)} \pi(\theta|x)$, where both $\pi, \pi_R$ are exponential family distributions?
2
votes
0answers
28 views

Help with Old exam questions on Bayesian Inference Problem [closed]

I've been trying to teach myself bayesian inference and I found a question sheet online ---> https://math.mit.edu/~dav/05.dir/ps6.pdf. I was attempting to solve question 4 but I'm not sure the method ...
0
votes
1answer
30 views

Is there an algebra error in this derivation in a Coursera class of the posterior of a Normal with a Student-distributed mean?

This is a screenshot from Coursera's class "Bayesian Statistics: Techniques and Models", Week 1, "Non-conjugate models" lecture (any one can audit the class and access the materials for free): This ...
0
votes
1answer
62 views

Bayesian predictions from posterior parameter distributions

I have two physical models $f(\theta)$ and $g(\theta)$ (not probability distributions) parameterized on the same set of parameters $\theta$. I also have data $y$ with measurement noise $\epsilon$ ...
1
vote
0answers
18 views

Multivariate bayesian inference: learnig about the mean of a variable by observing another variable

I want to derive a Bayesian learning procedure where I don't only learn from my own signal, but also from other signals which are correlated to mine. I thought it could simply work with Bayesian ...
1
vote
3answers
72 views

Bayesian Inference: Feeding Posterior back in as Prior

I've just started reading about Bayesian Inference, and one thing I've wondered about is if it's possible to feed the posterior in as a new prior for a new model, using the same data. Or would that ...
0
votes
0answers
10 views

Energy based learning for HMMs: Viterbi training

I understand why we want to maximise the posterior probability to find the most likely sequence of hidden variables but I've read that this is equivalent to minimising some concept of free energy. I'...
1
vote
1answer
64 views

Mechanics for combining likelihood and prior in non-trivial case

Bayes rule is simple enough on its face: $$ pr(B|A) = \frac{pr(A|B)pr(B)}{pr(A)} $$ If these things are known scalar probabilities, the answer is simple to compute. But I'm failing to understand ...
0
votes
1answer
16 views

MAP versus Component-Wise Maximum Marginal

Suppose I have the joint distribution: \begin{align} p(\mathbf{x}) = p(x_1, ... , x_n) \end{align} The maximum a posteriori (MAP) solution is given by: \begin{align} \mathbf{x}_{MAP} = \arg \max p(\...
0
votes
0answers
18 views

Relationship between posterior yielded from entire data vs posterior from subset of the data

I have a question about the relationship between a posterior for $\theta$ yielded from a pooled dataset of size $n$ and a posterior for $\theta$ yielded from a subset of the data, size $n_1$, and an ...
0
votes
0answers
8 views

Estimating the variance of an unknown vector

Imagine I have an MCMC approximation to the joint posterior distribution for the elements of some vector $\overrightarrow{A}$, and I want to estimate the sample variance of $\overrightarrow{A}$. For ...
1
vote
1answer
39 views

Posterior distribution from piecewise likelihood

Consider a hierarchical Bayesian model for analysing data from an inhomogeneous Poisson process that we observe in discrete time. Let $Y_i, i = 1,...,n$, be the number of events occurring in the time ...
0
votes
1answer
32 views

Is it valid to use samples drawn from posterior using MCMC as prior distribution in sequational updating?

Let's say there are two sets of parameters $x$ and $y$, and two sets of data $a$ and $b$. $x$ only depends on $a$, but $y$ depends on both $a$ and $b$. I first sample from the posterior $$P(x \mid a) ...
0
votes
1answer
18 views

Notation in definition of a quantity involving uncertainty and posterior probability

As a probs & stats noob, I'm still getting confused by notation. I would appreciate if someone could elaborate a bit on what's going on here, on p12 of Settles' Active Learning survey (2012). He ...
0
votes
0answers
16 views

Comparing posteriors when the same data are examined with different priors

I have two Bayesian regression models: both use the same data $D$ and all same specification $M$ but the different priors $p_1(\theta)$ and $p_2(\theta)$. Can I interpret the posteriors from the two ...
1
vote
1answer
45 views

Poisson-gamma posterior mean expectation

Let's have a gamma prior $\lambda\sim \operatorname{Gamma}(a,b)$ (mean: $\frac{a}{b}$) With Poisson data $Y\mid \lambda\sim \operatorname{Pois}(N\lambda)$ (mean: $N\lambda$) The posterior is $\...