Bayesian inference is a method of statistical inference that relies on turning the model parameters into random variables and applying Bayes' theorem to deduce probability statements about the parameters or hypotheses, conditional on the observed dataset.

learn more… | top users | synonyms

0
votes
1answer
11 views

“Multiple definitions of node p[1]” Error using WinBUGS

I have written the following code in WinBUGS and every time I try to compile the data after loading in the data I get the same error which is "Multiple definitions of node ...
3
votes
0answers
15 views

calculation threshold for minimum risk classifier?

Suppose Two Class $C_1$ and $C_2$ has an attribute $x$ and has distribution $ \cal{N} (0, 0.5)$ and $ \cal{N} (1, 0.5)$. if we have equal prior $P(C_1)=P(C_2)=0.5$ for following cost matrix: $L= ...
2
votes
3answers
92 views

A question on Bayesian updating (Han Solo and the Bayesian priors)

http://www.statslife.org.uk/the-statistics-dictionary/2148-han-solo-and-bayesian-priors in the preceding article, there is a bit of Bayesian updating that does not make sense to me. We have a prior ...
1
vote
0answers
12 views

Matching a query distribution to a family of template distributions

I was turning over a hypothetical question in my head: Suppose I have a set of template probability distributions, let's say each giving the probability of the occurrence of certain objects like ...
1
vote
0answers
43 views

Bayesian probability, verifying stupid examples

A wikipedia article on Bayesian inference offers an informal example in which a friend has had a baby, about which I know nothing more, and a (presumably reliable) mutual friend shows me a picture of ...
3
votes
0answers
49 views

Equal Prior Probablility and Linear Decision Boundary, a Simple Calculation Problem?

I get trouble in calculation on ML. I read one problem as follows: in a two class classification, with equal prior probability $P(C_1)=P(C_2)=0.5$ if the distribution of instance in classes be ...
1
vote
1answer
28 views

Gibbs Sampler - Sample mean convergence

To simulate from the posterior distribution $p(\theta|Y)$ where $\theta = (\mu,\lambda_1,\lambda_2)$, I run a Gibbs sampler to draw approximately random values from $p(\theta|Y)$. This Gibbs sampler ...
4
votes
1answer
28 views

Gibbs Sampler output: how many Markov chains?

When running a Gibbs sampler (for $n=200$ Iterations) with two full conditionals, I get the output $\mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}$. So $\mathbf{x}$ is the realizations of a Gibbs ...
1
vote
0answers
22 views

Multi-Level (Sparse) A/B Testing

I've been reading some articles about Bayesian A/B testing such as: http://engineering.richrelevance.com/bayesian-ab-tests/ but my application requires a framework for handling sparsity and group ...
0
votes
0answers
14 views

Calculating Marginal Data Density for VAR Model

I am currently estimating Bayesian vector autoregressive (BVAR) models and I would like to do model comparison with Bayes factors. I have read about the Gelfand-Day method, the Geweke (1999) modified ...
-4
votes
4answers
169 views

Jelly Donut Puzzle - What Can Statistics Say About These Donuts [on hold]

I give you a box of 12 donuts. You randomly remove 5. They are all jelly donuts. What is the probability that there is another jelly donut in the box?
2
votes
1answer
30 views

Full conditionals - Gibbs Sampler

i want to draw samples from a 5-dimensional posterior distribution $f(k,\theta,\lambda,b_1,b_2|Y=y)$. From Bayes-Theorem there is the following relationship between posterior and likelihood: ...
1
vote
1answer
44 views

MCMC/EM limitations? MCMC over EM?

I am currently learning hierarchical Bayesian models using JAGS from R, and also pymc using Python ("Bayesian Methods for Hackers"). I can get some intuition from this post: "you will end up with a ...
2
votes
1answer
35 views

Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large ...
3
votes
2answers
39 views

Bayesian Inference applied to a time series

I am trying to better understand how Bayesian Inference can be used. Let's assume I am measuring a given property, $\theta$, of some material. I have done 100 measurements. These had a Gaussian ...
4
votes
0answers
25 views

Making sense of standard deviation after sampling using Cholesky

I have an inverse problem with over 65,000 degrees of freedom. I am using Bayesian formulation to solve this problem. After using the optimization algorithm to obtain MAP solution, I want to calculate ...
1
vote
0answers
18 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
6
votes
0answers
68 views

Bayesian inference with sampling and mixture models

I'm having some trouble doing Bayesian inference on an experience I have in hands. I apologize in advance if it is too complex, but I couldn't find a trivial way to split it in several parts. Let ...
2
votes
1answer
32 views

When is mu_a used in this STAN example?

I'm looking at an example of a random effects model with 2 random effects fit by Peter Li demonstrating how get models fit in lmer into STAN. The code for this and the accompanying data are stored ...
0
votes
1answer
39 views

Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes ...
0
votes
0answers
3 views

Predicting with Posterior PMF(discrete non-pram dist) ? Predictive posterior distribution?

Sorry in advance if my question is abit awkward, I'm somewhat confused because most of the tutorials on the Internet mention that you should use Posterior-predictive dist to predict new data. The ...
11
votes
1answer
109 views

Why does adding a lag effect increase mean deviance in a Bayesian hierarchical model?

Background: I'm currently doing some work comparing various Bayesian hierarchical models. The data $y_{ij}$ are numeric measures of well-being for participant $i$ and time $j$. I have around 1000 ...
1
vote
0answers
25 views
+50

Factorizable time evolution in a dynamic stochastic process

I have a stationary dynamic system which at each given time $t$ is in state $x_t \in \mathcal{X}$. The set of states $\mathcal{X}$ is assumed to be finite but too large to be enumerated by a practical ...
2
votes
0answers
14 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
2
votes
0answers
47 views

Combining two probabilistic predictions

I am solving a machine learning task in which I need to predict a label $\tau$ from input $\vec x$. The input $\vec x$ can be considered as two parts $\vec u$ and $\vec v$ ($\vec x$ can be thought of ...
2
votes
1answer
37 views

Sampling Twice and Posteriors

I have a random variable with some unknown distribution with support over $[0, 1]$. Every turn, I sample a $p_t$ from this distribution. However, I am unable to observe $p_t$ directly. Instead I ...
-1
votes
0answers
11 views

Circumstances Under Which Conjugate Distributions Occur

How common is it for a posterior distribution P(x|a) to not be of the same distribution as P(x)? Are there certain areas there this is more likely to arise?
4
votes
1answer
87 views

Is a spike-and-slab prior a proper prior?

Is a spike and slab prior a proper prior? (I am talking about a (product of Bernoulli) spike and Normal slab) If not, does it still lead to a proper posterior?
3
votes
0answers
42 views

Understanding Chib (1998) Bayesian multiple changepoint model

Ungated link to the paper Chib, S. (1998). Estimation and comparison of multiple change-point models. Journal of Econometrics, 86(2), 221–241. doi:10.1016/S0304-4076(97)00115-2 The context of the ...
0
votes
0answers
28 views

How can we estimate the predictive interval in Lasso regression

Dear Community members, I am using lasso to solve an inverse problem (a Fredholm) which I can reframe as \begin{equation}\min_{\mathbf x ~~{\rm with}~~x_n\geq 0} \ell_{\rm Lasso}(\mathbf x, ...
0
votes
0answers
17 views

Posterior predicted distribution, practical question

I'm new here to this place but I have already learned so much here. Yet I still remain with a large question involving my thesis in econometrics and medical scoence. For a starter, I have read ...
0
votes
0answers
9 views

Reducing deviance by combining models

In this book from which I've screen-shotted from, Clarke gives an example of encompassing competing models to compare deviance score reductions. For reference, I have also asked about encompassing ...
1
vote
1answer
124 views

Machine Learning

I have been working on some self study "machine learning". Based on a few posts here, I wanted to make a program that "learned" via Bayes Law. I test it with some simple truth tables. It recalls the ...
3
votes
2answers
103 views

What to do when your likelihood function has a double product with small values near zero - log transform doesn't work?

I currently have a likelihood function defined as the following: $$ L=\prod_{i=1}^{N}\left[\prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_A)\times\phi + \prod_{s=1}^{S_i}L_{is}(y\space|\space ...
0
votes
1answer
41 views

K-mean clustering with unknown k

How do I perform k-mean clustering with unknown k? I also need to provide a confidence interval for k. I am thinking in the line of putting a Poisson prior on k. Does that make sense? Does there exist ...
7
votes
2answers
535 views

What frequentist statistics topics should I know before learning Bayesian statistics?

I was wondering if there is a subset of topics of frequentist statistics that one should know before starting to learn Bayesian statistics. Once I read that it seems that the two trends are ...
0
votes
0answers
11 views

How to estimate Mean and Variance of a Gaussian dataset of 20 numbers using ML,MAP and Bayesian Inference?

I have generated 20 random numbers from a Gaussian distribution with mean 5, and standard deviation 1. I have a question that has asked me to estimate the mean and variance using the above methods. ...
1
vote
0answers
49 views

how to construct the likelihood if my errors are not Gaussian

My aim is to study the correlation between 2 parameters knowing that I have measurement errors in both parameters, i.e. I have uncertainties on the independent and dependent parameters. I want to ...
0
votes
1answer
23 views

Why using vague (or noninformative) priors? [duplicate]

In my Bayesian class, we are always required to specify vague (or noninformative) priors for bayesian modeling. I am quite confused about that. If I understand correctly, the main advantage of ...
5
votes
2answers
136 views

What is a reasonable noninformative prior for quadratic and cubic coefficients in Bayesian polynomial regression?

Say we have a Bayesian polynomial regression like the following. $$y_i \sim N(\mu_i, \sigma^2)$$ $$\mu_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \beta_3 x_i^3 $$ where $x_i$ is some mean centred ...
0
votes
1answer
62 views

Maximum a posteriori estimation with one single training example?

I am doing maximum a posteriori (MAP) to estimate $\mu$ and $\sigma$ with $N$ samples drawn from $\mathcal{N}(5, 1)$. The priors that I place are $\mu\sim\mathcal{N}(5, 1)$ and ...
0
votes
1answer
60 views

What are the advantages of using a Bayesian neural network

Recently I read some papers about the Bayesian neural network (BNN) [Neal, 1992], [Neal, 2012], which gives a probability relation between the input and output in a neural network. Training such a ...
0
votes
0answers
17 views

Bayesian Ridge vs Stochastic Gradient Descent

I was running some Regression algorithms on a dataset and it just so happens, that the Bayesian ridge Regression techniques is performing not so well as the SGD (Stochastic Gradient Descent) ...
2
votes
0answers
30 views

Compound Poisson Distributions: When, Why, and How To Split the Problem

I've just stumbled upon the Compound Poisson Distribution (CPD) and it seems to be precisely what I need. For the purposes of this post, let's suppose I have a store that sells many items of ...
0
votes
0answers
8 views

Verifying posterior distribution is proper when integration is not feasible

I am wondering how we can verify the posterior distribution is proper if the expression we have for it is difficult (or impossible) to integrate. For example, I have a model that is based on a ...
1
vote
1answer
36 views

Use Bayesian modeling & PyMC to find players with similar skill levels to match for game

I'm trying to learn Bayesian modeling and PyMC and I'm working on building a match-making service for an online game that avoids pitting two extremely unequal opponents against one another. What I'd ...
4
votes
1answer
485 views

Formula for Bayesian A/B Testing doesn't make any sense

I'm using the formula from Bayesian ab testing in order to compute results of AB test using Bayesian methodology. $$ \Pr(p_B > p_A) = \sum^{\alpha_B-1}_{i=0} ...
0
votes
0answers
17 views

How is a most probable Hessian used to calculate Bayesian Prediction Intervals?

I'm using the MATLAB Neural Network Toolbox to use Bayesian Regularization (trainbr.m) to train a neural network which has one input, one hidden, and one output layer. I use this network for ...
1
vote
0answers
37 views

Are sufficient statistics for regression equivalent in the frequentist and Bayesian cases?

If I have a Poisson regression such that $\lambda = \alpha + \beta t$, $\alpha + \beta t \geq 0$ $\forall t, \alpha, \beta$ and $Y_t \sim \textrm{Poisson}(\lambda_t)$ for which I have 10 observations ...
0
votes
0answers
25 views

Type-II maximum likelihood (Empirical Bayes)

Given a categorical variable $X \sim Cat(K,{\bf p})$, where $K$ is the number of classes and ${\bf p}$ is a $K$-dimensional parameter vector, we can place a Dirichlet prior $Dir(\alpha)$ over ${\bf ...