Bayesian inference is a method of statistical inference that relies on turning the model parameters into random variables and applying Bayes' theorem to deduce probability statements about the parameters or hypotheses, conditional on the observed dataset.

learn more… | top users | synonyms

-1
votes
3answers
54 views

Jelly Donut Puzzle - What Can Statistics Say About These Donuts

I give you a box of 12 donuts. You randomly remove 5. They are all jelly donuts. What is the probability that there is another jelly donut in the box?
1
vote
1answer
24 views

Full conditionals - Gibbs Sampler

i want to draw samples from a 5-dimensional posterior distribution $f(k,\theta,\lambda,b_1,b_2|Y=y)$. From Bayes-Theorem there is the following relationship between posterior and likelihood: ...
1
vote
1answer
32 views

MCMC/EM limitations? MCMC over EM?

I am currently learning hierarchical Bayesian models using JAGS from R, and also pymc using Python ("Bayesian Methods for Hackers"). I can get some intuition from this post: "you will end up with a ...
2
votes
1answer
33 views

Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large ...
3
votes
2answers
32 views

Bayesian Inference applied to a time series

I am trying to better understand how Bayesian Inference can be used. Let's assume I am measuring a given property, $\theta$, of some material. I have done 100 measurements. These had a Gaussian ...
4
votes
0answers
20 views

Making sense of standard deviation after sampling using Cholesky

I have an inverse problem with over 65,000 degrees of freedom. I am using Bayesian formulation to solve this problem. After using the optimization algorithm to obtain MAP solution, I want to calculate ...
1
vote
0answers
16 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
6
votes
0answers
39 views

Bayesian inference with sampling and mixture models

I'm having some trouble doing Bayesian inference on an experience I have in hands. I apologize in advance if it is too complex, but I couldn't find a trivial way to split it in several parts. Let ...
0
votes
0answers
31 views

Bayesian Regression

Let's assume the following regression setting: $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon$ $\epsilon$ ~ $\mathcal{N}(0,\sigma_0^2)$ Suppose that $(x_1,x_2) \sim ...
2
votes
1answer
28 views

When is mu_a used in this STAN example?

I'm looking at an example of a random effects model with 2 random effects fit by Peter Li demonstrating how get models fit in lmer into STAN. The code for this and the accompanying data are stored ...
0
votes
1answer
29 views

Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes ...
0
votes
0answers
21 views

GLM/LM frequentist and bayesian model?

I am taking one class on bayesian method applied to ecology. According to my understanding, to estimate the parameters of GLM and LM model if we do not see the parameters as variables, then we can ...
0
votes
0answers
3 views

Predicting with Posterior PMF(discrete non-pram dist) ? Predictive posterior distribution?

Sorry in advance if my question is abit awkward, I'm somewhat confused because most of the tutorials on the Internet mention that you should use Posterior-predictive dist to predict new data. The ...
6
votes
0answers
61 views
+200

Why does adding a lag effect increase mean deviance in a Bayesian hierarchical model?

Background: I'm currently doing some work comparing various Bayesian hierarchical models. The data $y_{ij}$ are numeric measures of well-being for participant $i$ and time $j$. I have around 1000 ...
0
votes
0answers
13 views

Factorizable time evolution in a dynamic stochastic process

I have a stationary dynamic system which at each given time $t$ is in state $x_t \in \mathcal{X}$. The set of states $\mathcal{X}$ is assumed to be finite but too large to be enumerated by a practical ...
2
votes
0answers
14 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
2
votes
0answers
47 views

Combining two probabilistic predictions

I am solving a machine learning task in which I need to predict a label $\tau$ from input $\vec x$. The input $\vec x$ can be considered as two parts $\vec u$ and $\vec v$ ($\vec x$ can be thought of ...
2
votes
1answer
35 views

Sampling Twice and Posteriors

I have a random variable with some unknown distribution with support over $[0, 1]$. Every turn, I sample a $p_t$ from this distribution. However, I am unable to observe $p_t$ directly. Instead I ...
-1
votes
0answers
11 views

Circumstances Under Which Conjugate Distributions Occur

How common is it for a posterior distribution P(x|a) to not be of the same distribution as P(x)? Are there certain areas there this is more likely to arise?
4
votes
1answer
86 views

Is a spike-and-slab prior a proper prior?

Is a spike and slab prior a proper prior? (I am talking about a (product of Bernoulli) spike and Normal slab) If not, does it still lead to a proper posterior?
3
votes
0answers
37 views

Understanding Chib (1998) Bayesian multiple changepoint model

Ungated link to the paper Chib, S. (1998). Estimation and comparison of multiple change-point models. Journal of Econometrics, 86(2), 221–241. doi:10.1016/S0304-4076(97)00115-2 The context of the ...
0
votes
0answers
27 views

How can we estimate the predictive interval in Lasso regression

Dear Community members, I am using lasso to solve an inverse problem (a Fredholm) which I can reframe as \begin{equation}\min_{\mathbf x ~~{\rm with}~~x_n\geq 0} \ell_{\rm Lasso}(\mathbf x, ...
0
votes
0answers
17 views

Posterior predicted distribution, practical question

I'm new here to this place but I have already learned so much here. Yet I still remain with a large question involving my thesis in econometrics and medical scoence. For a starter, I have read ...
0
votes
0answers
9 views

Reducing deviance by combining models

In this book from which I've screen-shotted from, Clarke gives an example of encompassing competing models to compare deviance score reductions. For reference, I have also asked about encompassing ...
1
vote
1answer
124 views

Machine Learning

I have been working on some self study "machine learning". Based on a few posts here, I wanted to make a program that "learned" via Bayes Law. I test it with some simple truth tables. It recalls the ...
3
votes
2answers
99 views

What to do when your likelihood function has a double product with small values near zero - log transform doesn't work?

I currently have a likelihood function defined as the following: $$ L=\prod_{i=1}^{N}\left[\prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_A)\times\phi + \prod_{s=1}^{S_i}L_{is}(y\space|\space ...
0
votes
1answer
40 views

K-mean clustering with unknown k

How do I perform k-mean clustering with unknown k? I also need to provide a confidence interval for k. I am thinking in the line of putting a Poisson prior on k. Does that make sense? Does there exist ...
6
votes
2answers
529 views

What frequentist statistics topics should I know before learning Bayesian statistics?

I was wondering if there is a subset of topics of frequentist statistics that one should know before starting to learn Bayesian statistics. Once I read that it seems that the two trends are ...
0
votes
0answers
11 views

How to estimate Mean and Variance of a Gaussian dataset of 20 numbers using ML,MAP and Bayesian Inference?

I have generated 20 random numbers from a Gaussian distribution with mean 5, and standard deviation 1. I have a question that has asked me to estimate the mean and variance using the above methods. ...
1
vote
0answers
46 views

how to construct the likelihood if my errors are not Gaussian

My aim is to study the correlation between 2 parameters knowing that I have measurement errors in both parameters, i.e. I have uncertainties on the independent and dependent parameters. I want to ...
0
votes
1answer
23 views

Why using vague (or noninformative) priors? [duplicate]

In my Bayesian class, we are always required to specify vague (or noninformative) priors for bayesian modeling. I am quite confused about that. If I understand correctly, the main advantage of ...
5
votes
2answers
134 views

What is a reasonable noninformative prior for quadratic and cubic coefficients in Bayesian polynomial regression?

Say we have a Bayesian polynomial regression like the following. $$y_i \sim N(\mu_i, \sigma^2)$$ $$\mu_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 $$ where $x_i$ is some mean centred ...
0
votes
1answer
62 views

Maximum a posteriori estimation with one single training example?

I am doing maximum a posteriori (MAP) to estimate $\mu$ and $\sigma$ with $N$ samples drawn from $\mathcal{N}(5, 1)$. The priors that I place are $\mu\sim\mathcal{N}(5, 1)$ and ...
0
votes
1answer
59 views

What are the advantages of using a Bayesian neural network

Recently I read some papers about the Bayesian neural network (BNN) [Neal, 1992], [Neal, 2012], which gives a probability relation between the input and output in a neural network. Training such a ...
0
votes
0answers
16 views

Bayesian Ridge vs Stochastic Gradient Descent

I was running some Regression algorithms on a dataset and it just so happens, that the Bayesian ridge Regression techniques is performing not so well as the SGD (Stochastic Gradient Descent) ...
2
votes
0answers
29 views

Compound Poisson Distributions: When, Why, and How To Split the Problem

I've just stumbled upon the Compound Poisson Distribution (CPD) and it seems to be precisely what I need. For the purposes of this post, let's suppose I have a store that sells many items of ...
0
votes
0answers
8 views

Verifying posterior distribution is proper when integration is not feasible

I am wondering how we can verify the posterior distribution is proper if the expression we have for it is difficult (or impossible) to integrate. For example, I have a model that is based on a ...
1
vote
1answer
36 views

Use Bayesian modeling & PyMC to find players with similar skill levels to match for game

I'm trying to learn Bayesian modeling and PyMC and I'm working on building a match-making service for an online game that avoids pitting two extremely unequal opponents against one another. What I'd ...
4
votes
1answer
479 views

Formula for Bayesian A/B Testing doesn't make any sense

I'm using the formula from Bayesian ab testing in order to compute results of AB test using Bayesian methodology. $$ \Pr(p_B > p_A) = \sum^{\alpha_B-1}_{i=0} ...
0
votes
0answers
14 views

How is a most probable Hessian used to calculate Bayesian Prediction Intervals?

I'm using the MATLAB Neural Network Toolbox to use Bayesian Regularization (trainbr.m) to train a neural network which has one input, one hidden, and one output layer. I use this network for ...
1
vote
0answers
36 views

Are sufficient statistics for regression equivalent in the frequentist and Bayesian cases?

If I have a Poisson regression such that $\lambda = \alpha + \beta t$, $\alpha + \beta t \geq 0$ $\forall t, \alpha, \beta$ and $Y_t \sim \textrm{Poisson}(\lambda_t)$ for which I have 10 observations ...
0
votes
0answers
24 views

Type-II maximum likelihood (Empirical Bayes)

Given a categorical variable $X \sim Cat(K,{\bf p})$, where $K$ is the number of classes and ${\bf p}$ is a $K$-dimensional parameter vector, we can place a Dirichlet prior $Dir(\alpha)$ over ${\bf ...
4
votes
2answers
94 views

Calculating marginal distribution via integration

Suppose that we have an IID random sample $\mathbf{x} = (x_1, \dots, x_n)$ from a given distribution with the following PDF: $$\theta (1 - e^{-x})^{\theta -1}e^{-x}, \, x > 0, \, \theta > 0$$ ...
2
votes
2answers
96 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
0
votes
0answers
40 views

Constructing Bayes Factor comparing a single correlation coefficient in the entire sample vs. separate coefficients by group?

I have a sample consisting of 4 independent groups of participants. For any two variables within the sample, I would like to quantify the improvement in fit to the data from estimating a separate ...
0
votes
0answers
11 views

Trying to explain your data: model the noise or filter it?

I can think of two ways to evaluate a model given noisy data: Calculate a distance between your model and your data, and use it within a noise model to calculate the probability of the data. E.g. ...
0
votes
0answers
39 views

recommended books for preliminary concepts of Bayesian Statistics [duplicate]

I am interested in learning Bayesian Statistics related for my CS career. I have a very little background on frequentist statistics, which I know differs from the bayesian approach. The reason of my ...
2
votes
0answers
26 views

Bayesian Linear Model Posterior as Sum of Squares?

As part of a homework, I am asked to do the math from the Normal-Inverse Gamma linear regression model. Starting from priors $N(\beta_0, \sigma^2 A)$ and $IG(\alpha_0, \delta_0)$ and with the help ...
0
votes
0answers
13 views

When to pick Scale or Rate for Gamma/ Inverse Gamma parameters? Picking up the correct conjugate prior

I am very confused about the following problem. I think my question is both theoretical and applied. When I model a Gamma distribution, when do I use this model, so I guess Inverse Gamma: ...
0
votes
2answers
56 views

(Failure) probability calculation

I am working on mortality in 12 hospitals performing cardiac surgery in babies. The dataset is available here: Surg dataset. The dataset is structured in this way: ...