Questions tagged [bayesian]

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

Filter by
Sorted by
Tagged with
2
votes
1answer
23 views

Semi-conjugate inverse Wishart posterior, can we obtain the marginal?

In Hoff's text (A First Course in Bayesian Statistical Methods), he uses a semi-conjugate inverse-Wishart prior for the covariance matrix of a multivariate normal process. In equation 7.9, he has the ...
5
votes
2answers
1k views

Calculating Bayes Factor from a correlation coefficient

I'm wondering whether anyone knows whether it is possible to directly calculate a Bayes Factor (comparing null model of zero correlation to non-zero correlation) given just a correlation coefficient ...
4
votes
1answer
148 views

Neural Networks - Strategies for problems with high Bayes error rate

I am building a Neural Network for a binary classification problem where the Bayes error (lowest possible error rate) is probably close to 50%. What makes the task easier is that I don't need to make ...
1
vote
0answers
14 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
0
votes
0answers
7 views

How would I use Evan Miller's sort criterion to modify the ranks of Bayesian average ratings?

Evan Miller wrote these guidelines for constructing a Bayesian average ratings and then sorting them using a multi-linear loss function: http://www.evanmiller.org/bayesian-average-ratings.html I ...
2
votes
0answers
28 views

Parameter estimation by averaging over all high-likelihood possibilities?

I am refereeing a chemistry paper. The authors are trying to interpret some experimental data by comparison with numerical simulations. They have run many simulations using different combinations of ...
1
vote
0answers
22 views

Why do non-informative a priori distributions be chosen to compare the Bayesian and frequentist estimation method?

For example for GARCH models $$\sigma_t^2=\alpha_0 +\alpha_1 y_{t-1}^2 + \beta_1 \sigma^2_{t-1}$$ it is usual to use as distributions for the parameters of truncated normal distributions with very ...
1
vote
1answer
19 views

Unbiasedness of Bayesian Posterior Mean Under Bayesian and Frequentist Models [duplicate]

This is an extension to this previous question, and is related to exercise 4.7 from Gelman et al.'s BDA3. When is the Bayesian posterior mean $m(y) \equiv E[\theta \mid y]$ unbiased for $\theta$, ...
1
vote
1answer
38 views

Treating missing data in making Bayesian inference

Suppose we have two biased coins $X_1,X_2$ that are possibly correlated to each other. In each round, when both the coins are tossed, there can be four possible outcomes: $(HH,HT,TH,TT).$ Let's ...
1
vote
0answers
13 views

Can I use a bayesian spatio-temporal model in cluster areas I chose from local Moran's I?

I have crime rates for municipalities in a state with hourly frequency. I want to make predictions about the spatio-temporal behavior of that variable. Is it possible to run a local Moran's I to ...
3
votes
2answers
70 views

Pattern Recognition and Machine Learning (Bishop) - How is this log-evidence function maximized with respect to $\alpha$?

In the book Pattern Recognition and Machine Learning, the author writes the log-evidence function (equation 3.86 in page 167): ln $p(\textbf{t}| \alpha, \beta) = \frac{M}{2}$ ln $\alpha$ + $\frac{N}{...
2
votes
1answer
264 views

Bayes optimal decision for logistic regression: Self-study exercise

We want to find the Bayes optimal decision for logistic regression. That means that the goal is to find the actions, which minimize our expected loss (also often called expected cost or risk). Here ...
4
votes
2answers
38 views

What is the relation between “conjugate priors” and the approximate inference?

I know that "conjugate prior" is to help us calculate the the denominator of the Bayes formula(to make the calculations easier). And I just learnt to approximate the inference by mean field ...
0
votes
1answer
44 views

Calculating function of posterior distribution of Gaussian?

This is a problem from chapter 11 of All of Statistics. The question is as follows: Let $X_1,\ldots,X_n \sim N(\mu,1)$. Let $\theta = e^\mu$. Find the posterior density for $\theta$ analytically and ...
0
votes
2answers
25 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
3
votes
1answer
52 views

Bayesian inference about means, observing only the sum of two random variables

I have: $X \sim \mathcal{N}(\mu_x, \sigma_x^2)$ and $Y \sim \mathcal{N}(\mu_y, \sigma_y^2)$. $X$ and $Y$ are independent. $\mu_x$ and $\mu_y$ are not known and I want to learn about them (Bayesian ...
1
vote
2answers
3k views

Showing that the maximum likelihood estimator (MLE) exists but is not unique

I have a few questions with regards to a solution to the problem below: How is it possible to have $max_{1\leq{i}\leq{n}}x_i-1<\theta<min_{1\leq{i}\leq{n}}x_i$? How can a value of $\theta$ be ...
3
votes
1answer
38 views

Motivations for experiment design in statistical learning?

My interests in statistics centre around statistical learning, including Bayesian inference, inference in combinatorial spaces, Monte Carlo methods, Markov decision processes, modeling stochastic ...
0
votes
1answer
23 views

pymc3: Updating the standard error prior

I am estimating a Bayesian multiple regression using continuous data on both the dependent variable and the regressors. My goal is to iteratively estimate the coefficient distributions as more data ...
0
votes
0answers
24 views

How to use Bayes' rule to updating from initial prior probability to final posterior probability?

For example, I update from a posterior probability P(H|D1) to a new posterior probability P(H|D1,D2) given a second piece of data D2 is: P(H|D2, D1)=P(H|D1)P(D2|H, D1)/P(D2|D1) #Equation1 Now I'm ...
1
vote
1answer
25 views

Conditional probability of tossing coins with uncertain head probability

Suppose there are two coins A and B. When tossing a coin $i$, "head" happens with probability $p_i$. The problem is that $p_i$ itself is a random variable. Say that the associated probability ...
1
vote
1answer
59 views

Probability of a box containing a combination of color

Let's say, we have a box containing 3 balls in it, they can be either red or blue. Someone draw a ball 5 times with replacement and get 4 red and 1 blue (not necessarily in order). Do you know how to ...
1
vote
0answers
50 views

Minimax estimators in Bayesian analysis

I got recently introduced to minimax methods in statistical decision theory. Is there an analogue in Bayesian analysis and some resources related to this?
0
votes
0answers
17 views

How to create an R vector to represent a log-uniform prior distribution? [closed]

how can I create an R vector to represent a log-uniform prior distribution? where the probabilities are proportional to 1/β. Thanks ...
0
votes
0answers
18 views

Proposal for correlation matrix with LKJ prior

I am writing a Gibbs sampler from scratch. As recommended in various places (http://www3.stat.sinica.edu.tw/statistica/oldpdf/A10n416.pdf, and in another question Covariance matrix proposal ...
2
votes
0answers
25 views

Help with Old exam questions on Bayesian Inference Problem [closed]

I've been trying to teach myself bayesian inference and I found a question sheet online ---> https://math.mit.edu/~dav/05.dir/ps6.pdf. I was attempting to solve question 4 but I'm not sure the method ...
1
vote
1answer
39 views

Combining multiple predictions with Bayes Theorem

I have multiple weather forecasters who each use their own unique, independent calculation for prediction of the weather for the next day. We are only concerned with rain predictions to know if we may ...
2
votes
1answer
40 views

Metropolis Hastings proposal for one parameter restricted to less than the other

Suppose I have parameters $\theta_0$ and $\theta_1$ with prior $$ p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$ that is, $\theta_0$ is less than $\theta_1$. The distributions ...
2
votes
0answers
21 views

Low Rank Gaussian Process vs Bayesian Linear Regression

A main benefit of Gaussian Process Regression is, that we not only get a prediction, but also a variance that we might use as indication of the prediction confidence. While bayesian linear regression ...
1
vote
0answers
49 views

Why do the non-informative a priori distributions give better results than the frequentist estimate?

For example, in the specific case of Markov-Switching GARCH models why is a non-informative prior distribution chosen for GARCH models with Bayesian estimation and why is this approach better than the ...
2
votes
0answers
28 views

Two sample test for equality of 2 dimensional distributions

I have a large sample from a 2 dimensional continuous unknown distribution. From that sample I could compute any data structure I need to hold an approximation of the sample distribution. This will be ...
1
vote
1answer
67 views

Variance of evidence lower bound(ELBO) loss function

When using Bayesian optimisation in a neural network our loss function is equal to: Here the first term is the KL divergence between the approximate and true posteriors. The second term is the ...
0
votes
0answers
27 views

Which statistics departments in the US are frequentist based and which are Bayesian? [closed]

I was just wondering if someone with a bit more insight in the "community" of statistics could give me a raw overview over the statstics departments of the most important Universities in the US (and ...
0
votes
0answers
13 views

how to use depended / non-random observations when trying to inference exponential parameter

consider this case: There is a price rate for a certain product that changes throw time, The price rate is changed every x minutes (unknown, not constant). This price has depended / non-random ...
7
votes
2answers
1k views

Posterior predictive distribution vs MAP estimate

Consider a training dataset $X$, a probabilistic model parameterized by $\theta$, and a prior $P(\theta)$. For a new data point $x^*$, we can compute $P(x^*)$ using: a fully bayesian approach: the ...
0
votes
0answers
27 views

fit a model to data

I want to fit a model to a data set, however each point is actually a distribution (i.e. I have the samples for each distribution). In an ideal world, I would assume that the distributions are ...
2
votes
2answers
33 views

Interpreting mixture of Gaussians (Variational Inference)

I've recently stated reading about mixture models and variational inference in this excellent paper, but I'm having troubles dissecting the models described, and have a couple of questions. Please see ...
1
vote
1answer
43 views

Intercept in a Bayesian model with categorical predictors (with brms)

I have a Bayesian logistic model fitted in R with brms. The predicted variable is binomial, the predictors are categorical. The model uses bernoulli family and a ...
0
votes
1answer
59 views

Bayesian predictions from posterior parameter distributions

I have two physical models $f(\theta)$ and $g(\theta)$ (not probability distributions) parameterized on the same set of parameters $\theta$. I also have data $y$ with measurement noise $\epsilon$ ...
14
votes
2answers
584 views

Hamiltonian monte carlo

Can someone explain the main idea behind Hamiltonian Monte Carlo methods and in which cases they will yield better results than Markov Chain Monte Carlo methods ?
2
votes
0answers
24 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
0
votes
0answers
43 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
0answers
22 views

Likelihood, posterior, prior interpretation and credibility/confidence_level with bayesian/frequentist approaches

This question was originally posted on physics exchange but one advised me to transfer it here. I try to understand the following article : testing general relativity from curvature and energy ...
0
votes
0answers
6 views

What kind of a priori distribution for the Markov Switching models?

Why in the Markov-Switching models is chosen as prior distribution for the probability of the transaction as follows: $$f(P) \propto \prod_{i=1}^K \left(\prod_{j=1}^K p_{i,j}\right) I \left\{0 < ...
1
vote
1answer
38 views

Geometric distribution with a capped number of trials - finding expectation and prior predictive distribution

So I am modeling a random variable which follows a geometric distribution with probability $\theta$ except that the total number of trials is capped at some value $n$. I.e., the probability mass ...
0
votes
1answer
968 views

Confusion Matrix to Calculate Probability

I was asked a relatively simple problem and was curious as to how to solve. Say I had a bomb detector at the airport, and it is 99.99% correct. That being, when the detector goes off or does not go ...
1
vote
0answers
22 views

Modeling Bayesian inference using Dirichlet conjugate

I'm trying to formalize my research question and want to know whether the following set up makes any sense or not. Suppose there are two coins $a$ and $b$. Probability of tossing heads are given by $...
0
votes
1answer
151 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
1
vote
1answer
204 views

How to calculate confidence intervals for linear mixed effects models when default methods with default settings fail?

I have a simple linear model describing a set of straight lines and would like to estimate confidence intervals for the parameters and the covariance matrix describing the hidden parameters. First ...
1
vote
1answer
19 views

Bayesian repeated updates, likelihood functions with different nature

Let's say we have a prior probability of some diseases 'D'. Then we have some data and likelihood function of symptoms (S) P(S|D) and we update priors. Then we have age (A) likelihood function P(A|D) ...