Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

0
votes
0answers
12 views

Probability of first time to an event

Suppose that $f_t$ is the probability density that a particular event happens at time $t$. For example, $f_t$ can be the probability density that a bus arrives at time $t$. Note that: $f_t$ is not a ...
1
vote
1answer
16 views

Admissible Bayes Rule

In the following wikipedia entry https://en.wikipedia.org/wiki/Admissible_decision_rule it is written that "Bayes rules with respect to proper priors are virtually always admissible" What do they ...
0
votes
0answers
8 views

Conditional posteriors - Is this the correct way to obtain them?

Forgive the novice question but I'm new to Bayesian inference... Suppose $y \sim p(y|\mu, \sigma, \theta)$ and we want to sample from the posterior using Gibbs sampling. That is, we need the ...
1
vote
0answers
29 views

When is the posterior distribution equal to the prior?

So I have heard that if the prior distribution is in the subexponential class, applying Bayes rule does not change the belief. I have been trying to find an example of this but I am unable to do so. I ...
0
votes
0answers
18 views

Efficiency of Bayesian Estimator

In the Wikipedia article for Minimum Mean Square Error https://en.wikipedia.org/wiki/Minimum_mean_square_error the Bayesian estimator is referred as Asymptotically Efficient using the similar ...
1
vote
0answers
18 views

How to derive conditional posterior of $\beta$ given

Define $y_i$ as following a multivariate normal distribution $$y_i \sim N(\mu + \beta x_i, \Sigma)$$ And suppose we have observations $(y_1,\dots,y_n,)$ and where $x_i$ are known observed $1$ ...
0
votes
0answers
9 views

Maximum likelihood deviance = lowest Bayesian deviance?

I've just run a logistic regression using the standard frequentist maximum likelihood approach and then again using Bayesian MCMC (weak priors, all ~ $n(0, 100)$). I calculated the deviance for each ...
0
votes
0answers
9 views

Is it possible to build a hierarchical pooling beta-binomial model using extra features/regressors?

If we consider the canonical partial pooling "batting rate" problem as soon here https://docs.pymc.io/notebooks/hierarchical_partial_pooling.html, is it possible to formulate the problem such that we ...
0
votes
0answers
17 views

Questions about approximate inference and calculating the posterior predictive

As I understand, computing the exact posterior of parameters $p(\theta|x)$ is nearly always impossible since we need to compute the evidence $\sum_\theta p(x|\theta)p(\theta)$ with every possible ...
1
vote
1answer
48 views

Monte Carlo maximum likelihood vs Bayesian inference

I recently heard about MCMLE (Monte Carlo maximum likelihood estimation) for finding $$ \hat\theta = \underset{\theta}{\text{argmax}} \frac{\exp\left(\theta^TT(y)\right)}{c(\theta)} $$ when the ...
1
vote
0answers
16 views

Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps: Probability Density Function The ...
7
votes
2answers
99 views

Are products of exchangeable RVs exchangeable?

Assume that $$X=(X_1, ..., X_n),: (\Omega, A,P)\to (\{0,1\}^n, 2^{{\{0,1\}}^n})$$ and $$Y=(Y_1, ..., Y_n):(\Omega, A,P)\to (\{0,1\}^n, 2^{{\{0,1\}}^n})$$are two random Variables that have binary RVs ...
0
votes
1answer
23 views

Variational Inference: Ising Model

I am self learning Variational Inference. Currently I am reading the chapter 21 book from Murphy 1 and trying to understand the Ising model (21.3.2). The Ising model here is used as denoising ...
0
votes
1answer
24 views

Two priors on the same parameter?

I received a text where the author was employing Stan language in order to show how to create a random walk with normally distributed parameters. His model had parameters $\mu_{t}$, ($1,2,...,T$), ...
49
votes
3answers
13k views

XKCD's modified Bayes theorem: actually kinda reasonable?

I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "...
1
vote
1answer
27 views

ABC: Population Monte Carlo (PMC) convergence statistics?

I'm using the abcpmc code: Approximate Bayesian Computing (ABC) Population Monte Carlo (PMC) implementation based on Sequential Monte Carlo (SMC) with Particle Filtering techniques. described in ...
0
votes
1answer
58 views

Bayes Estimator

Wikipedia has a section on the Bayes Estimator. https://en.wikipedia.org/wiki/Bayes_estimator Isn't Bayes Estimator simply the value of the parameter that minimizes the expected loss of a loss ...
0
votes
0answers
41 views
+50

Deriving predictive distribution

In Bayesian Regression, I am confused how to to get $f*$ and $\sigma*$, given $$y^∗ \mid \vec{y}\sim\mathcal{N}(f^∗ , σ^∗ )$$ $$ p(y^* \mid \vec{y}) = \int{p(y^* \mid \vec{w}) p(\vec{w} \mid \vec{y})...
0
votes
0answers
23 views

Some pointers on constructing a proposal density for a Metropolis-Hastings Algorithm [on hold]

I have a multidimensional likelihood, some parameters are covariance matrices. I'm looking for general pointers on constructing a proposal density for a Metropolis-Hastings Algorithm, from a ...
2
votes
0answers
20 views

When does knowing the causal structure of the data generating process improve supervised learning?

Consider a supervised learning prediction task where we have some real-valued feature vector X and wish to train a model that predicts discrete class label Y. When the model is deployed, Y will be ...
0
votes
1answer
22 views

Survival time problem exponential with gamma prior

The survival times, in days, of patients diagnosed with a severe form of a terminal illness are thought to be well modelled by an exponential($\theta$) distribution. We observe the survival times ...
0
votes
1answer
36 views

Obtaining posteriors for multivariate Normal mixture models

So I want to fit a mixture model $$f(y) = \pi_1 f_1 (y) + \pi_2 f_2 (y)$$ where $\pi_k = P(S = k)$ and $S_i$ is a latent unobserved variable. I assume that, conditional on $S=k$, we have the model ...
1
vote
0answers
23 views

How to choose an appropriate variational distribution?

I work in deep learning research and I am trying to learn how to use variational inference in order to approximate a posterior over the learned weights. I have looked extensively at Yarin Gal's ...
0
votes
3answers
84 views

Conjugate prior, unclear definition

Consider the following definition: A family $\cal F$ of probability distributions on $\Theta$ is said to be conjugate (or closed under sampling) for a likelihood function $f(x|\theta)$ if for every $\...
3
votes
1answer
52 views

Understanding bayesian model code from chapter 4 of “Statistical rethinking”

I'm trying to learn bayesian statistics from "Statistical rethinking" by Richard McElreath. In chapter 4, a model with Gaussian distribution of heights is introduced: $h_i \sim N(\mu, \sigma)$ $\mu \...
0
votes
0answers
46 views

Comparing 2 Bayesian Models with different structure

I'm a bit new to Bayesian statistics so please bear with me if this question is trivial. Let's say I have $100$ observations for $2$ Bernoulli variables $X$ and $Y$. I notice that they have the ...
0
votes
1answer
18 views

Bayesian hyperparameter learning in a multi-ouput Gaussian Process Regression

Let's imagine I have the following equation $y_t=f(x_t)+e_t$, where $f(x)$ follows a gaussian process, and $e_t\sim N(0,\Sigma)$. How does one go about to learn the hyperparameters, i.e., $\Sigma$ ...
1
vote
0answers
13 views

How to understand the following Bayesian schema?

My knowledge of probability is basic, and I understand the point of Bayesian interpretation most roughly. The following is part of this paper. It is about how p can be rational for person 1 and not-p ...
-1
votes
0answers
21 views

Probability calculation for posterior information [closed]

Assume there are 400 athletes in a training camp, who are required to attend the morning drill starting at 4 am. The attendance in morning drills is 70%, i.e. on an average, 280 athletes are present. ...
0
votes
0answers
7 views

Using the Expected value of the log as a score for the anomaly detection instead of just the expected value

While dealing with anomaly detection using a probabilistic model I need to compute the probability of an example coming out of the model I built. More specifically: If $p(X)$ is the model I built and ...
1
vote
0answers
8 views

How to interpret forestplot with pymc on standard devisions of two groups

I'm using pyMC3 to do Bayesian estimation supersedes the t test (BEST) and I was wondering how to actually interpret this result. I see both groups have significantly different stds because the bar ...
2
votes
1answer
64 views

Conjugate priors for dynamic model $x_{t+1}=Ax_{t}+\eta_t$

What conjugate priors do we have for the model(multivariate) $x_{t+1}=Ax_{t}+\eta_t$, where $\eta_t\overset{iid}\sim N(0,\Sigma)$? I was thinking of using $\tilde{x}=Diag[x_1,...,x_{n-1}]$, $\tilde{y}...
2
votes
1answer
39 views

How worried should I be about low acceptance rate in cold chain (parallel tempering MCMC sampler)

I have a very noisy/multimodal likelihood function for a 6-parameter model. The popular emcee sampler fails miserably (no matter how many chains I use and for how ...
4
votes
0answers
66 views

What was the fundamental error of Bayesian statistics according to Fisher?

Fisher wrote: "the theory of inverse probability is founded upon an error, and must be wholly rejected" I wonder what was Fisher's reasoning and what error he means in particular. The quote is ...
0
votes
2answers
50 views

PDF of incomplete beta function

I have a (probably) biased coin. I want to make a Bayesian inference on $\lambda=p(coin=\text{heads})$. I define my prior $p(\lambda)\sim Beta(\lambda; a=1,b=1)$. I toss the coin and update my ...
0
votes
0answers
5 views

How to use Pymc3 to create a Uniform and Von Mises mixture model [closed]

I am new to Pymc3 and currently trying to do an parameter estimation with it. I have a set of data which is assumed as a mixture of unform distribution and Von-mises. I found that the available ...
8
votes
2answers
328 views

Can a proper prior and exponentiated likelihood lead to an improper posterior?

(This question is inspired by this comment from Xi'an.) It is well known that if the prior distribution $\pi(\theta)$ is proper and the likelihood $L(\theta | x)$ is well-defined, then the posterior ...
0
votes
0answers
14 views

What is the relation between the effective sample size $n$ and the model dimension (the effective size of parameters) $p$ in Bayesian model selection?

What is the relation between the effective sample size $n$ and the model dimension (the effective size of parameters) $p$ in Bayesian model selection? Or is there any articles talking about this? I ...
0
votes
3answers
102 views

what does p( y | μ,σ²) really mean?

Just started to study Bayesian Statistics. I am very confused the concept of having a conditional probability on a distribution. Specifically: I understand what p( A | B ) where A="I am sick" and ...
0
votes
0answers
19 views

Calculating Cauchy prior

I've recently used the package BayesFactor in R with the default priors scale r. I have been advised to adjust the Cauchy width based on some pilot data rather than ...
2
votes
1answer
90 views

Proper prior leading to improper posterior

Preface I must say I am aware of previous discussions (e.g. this one) and also of this excellent, didactic proof using Fubini's theorem as presented by Jared Niemi [I'm not saying Jared Niemi is the ...
0
votes
0answers
13 views

SMOTE in Bayesian Networks

Oversampling or SMOTE is useful when the data is imbalanced. Here is the question I cant find the answer: Since we are dealing with probabilities in Bayesian Networks (probabilistic graphical model), ...
0
votes
1answer
25 views

What can I conclude about the distribution of wrong phone numbers?

Let's say I have a list of 100 phone numbers. I call them all. Nobody picks up for 70. I get someone on the line for 30. Of those, 10 are wrong numbers. What can I conclude about the distribution of ...
3
votes
1answer
32 views

Diffuse priors Bayes Factor

In textbooks I always read that it is necessary to have a proper prior on the parameter that we want to test with Bayes factor, otherwise we would always posteriori favor the model with less ...
1
vote
1answer
23 views

Objective function of Bayesian Model Averaging

I am quite confused about the objective function of the bayesian model averaging in the paper "Bayesian Averaging of Classifiers and the overfitting Problem".1 On the section 2, here is the first ...
0
votes
0answers
18 views

Is there any reference about the almost sure convergence of a posterior distribution to the posterior with non-informative prior?

I'm trying to show $$ \pi(\theta_n|X) \overset{a.s.}{\to} \pi(\theta|X) $$ where $\theta_n \sim N(0, n)$, and $\pi(\theta) = 1$(improper), and $X$ is normal. Is there any reference, or hint to ...
1
vote
1answer
74 views

If Bayesian approaches are better than frequentist then how can it be as practical?

In a textbook Probability Theory: The Logic of Science written by E. T. Jaynes and others, on page 13 it reads that: For many years, there has been controversy over ‘frequentist’ versus ‘...
2
votes
0answers
13 views

Appropriate Distribution for Diagonal Covariance Matrices

Let's say I have a model like: \begin{align} X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\ \mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\ \Sigma_X\mid \Psi, c &\sim \...
0
votes
0answers
8 views

Classication given several softmax probabilities

I am classifying images over time in categories such as office, bathroom, living room, etc. using CNNs (in this case VGG-16). The idea is to use all room categories' confidences (softmax probabilities)...
1
vote
0answers
103 views

Sampling Bayes factors under the null hypothesis to estimate a threshold of “significance” for hypothesis testing

Context: I have a psychology experiment with a 2 x 2 design (with Condition (label, no label) and ContrastType (head, tail) as ...