Questions tagged [prior]

In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior distribution represents a greater degree of information.

Filter by
Sorted by
Tagged with
3
votes
2answers
63 views

Accounting for uncertain information (few observations) in a prior (empirial Bayes)

I did not really know how to choose an adequate title for this question, so please feel free to change it. I have a weird case wherein frequentist and Bayesian philosophies come together. I am ...
0
votes
0answers
17 views

What is the point of using an uninformative prior distribution? [duplicate]

If I am completely unknowing as to the true value of $\theta$ in some parameter space $\Theta$, why would I use a flat prior distribution when all the information about $\theta$ will be in my sample? ...
0
votes
2answers
42 views

Can an improper prior distribution be informative?

I have just worked through an example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator, leading me to believe that improper priors are uninformative. But ...
0
votes
0answers
8 views

Finding prior conjugate for reparametrized model

Let $X_i$ be iid Bernoulli$(\pi)$ for $i=1,...,n$. My task is to find the prior conjugate for $\theta$, where $\theta$ is the natural parameter of the sampling model. The sampling model can be ...
18
votes
1answer
757 views

What are the best ways to generate Bayesian prior estimates using beliefs of non-statisticians?

I work with a lot of qualitative researchers and designers. Many of whom interact with users and develop strong, often accurate intuitions about how the data should look. I frequently try to quantify ...
0
votes
0answers
18 views

Sampling a proposed value with a limited range target when running MCMC [duplicate]

I want to do an MCMC algorithm and need to sample a proposed value from a proposed distribution. In the Metropolis algorithm, people usually use a normal distribution as proposal. But if the prior ...
1
vote
0answers
13 views

Linear regression - Bayesian Predictive distribution

I am trying to answer a question about linear regression but i am stuck: $y=w \cdot x + \epsilon, \epsilon \sim N(0,\alpha)$ i am also given a prior: $w\sim N(0,\beta)$ from which i was able to ...
3
votes
1answer
37 views

What prior would lead to $\ell_\infty$ regularization of model weights?

Gaussian prior on weights of a GLM lead to Ridge / $\ell_2$ squared regularization. Laplace prior leads to $\ell_1$ regularization Question What prior would lead to $\ell_\infty$ regularization ?
1
vote
1answer
43 views

Random-walk prior with ridge-like regularizarion?

I am working with a model that contains a large number of coefficients, arranged in an ordered vector $\beta_1, \dots, \, \beta_N $. I have some prior knowledge that could be used to improve the ...
0
votes
0answers
7 views

Sampling prior covariance matrices - nested sampling

I am trying to fit a multivariate Gaussian with a non-diagonal covariance matrix $\Sigma$ using nested sampling. Usually, in other Bayesian analyses, we would use a Inverse Wishart or LKJ prior on ...
1
vote
0answers
22 views

Bayesian inference with simple models and many data points: impact of priors and the number of data points

The setting. Let us assume I would like to perform Bayesian inference for a low-dimensional model on a large dataset, i.e., there are many more data points than there are parameters to identify. Let ...
0
votes
1answer
28 views

How to select variables when using shrinkage priors?

I am fitting a linear regression model using shrinkage priors (Horseshoe and Laplace/LASSO). This shrinks many of the variables close to zero, but I would like to select the variables. Can I use the ...
1
vote
0answers
24 views

Why is Half-Cauchy, Half-Student-t as prior for variance parameters better than a normal distribution?

Gelman often refers to using half-cauchy or half-student-t distributions for variance parameters. Why is it better than using a vague normal distribution such as N(0,10)? Can somebody explain me the ...
0
votes
0answers
25 views

State of the art (2019) of spike and slab priors?

Spike and slab priors for variable selection https://www.tandfonline.com/doi/abs/10.1080/01621459.1993.10476353 were initially proposed using a combination of normals prior and a mass at zero. After ...
2
votes
1answer
80 views

Why in Hamiltonian MCMC do we multiply the posterior distribution by the likelihood?

So maybe I am misunderstanding what the author is staying, but I am reading Chapter 14 of Kruschke's Doing Bayesian Analysis. I am reading about the software Stan and how it uses the Hamiltonian MCMC ...
0
votes
0answers
31 views

Setting a prior for zero-one-inflated beta bayesian regression

This is my first attempt with Bayesian statistics. I need to figure out if I have enough information to set more informative priors for a zero-one-inflated beta regression in ...
1
vote
0answers
26 views

What do these equations on Bayesian regression (MAP) from Chapter 3.3 in PRML by Bishop mean?

This was taken from Ch 3.3 on Bayesian Linear Regression from Pattern Recognition in Machine Learning by Bishop. Apparently the posterior can be described by eq 3.49. Eq 3.48 represents the prior ...
1
vote
1answer
31 views

Explanation of Equation 5.3 from Gaussian Processes for Machine Learning

I am currently reading through C. E. Rasmussen & C. K. I. Williams' Gaussian Processes for Machine Learning and was going through chapter 5. I could not exactly understand the derivation of ...
0
votes
0answers
24 views

Find posterior distribution given prior distribution

Problem Be $X_1,...,X_n$ a random sample of $X$ ~ $Geometric(\theta)$, i.e., $f(x|\theta)=\theta(1-\theta)^x \forall x = 0,1,2,...$ Assuming a prior distribution for $\theta$ find the posterior ...
0
votes
0answers
11 views

Deriving full conditional of ordered probit model (Bayes)

I have a question regarding the following exercise: I am able to compute the complete (full) data likelihood function, the full conditionals of $y^{*}_i$ and $\beta$. However, I do not know how to ...
3
votes
2answers
447 views

Shrinkage priors

I am building a Bayesian model where I to put shrinkage priors such as spike and slab, horseshoe prior, etc on some parameters for feature selection, but I am not able to decide which one is the best. ...
1
vote
1answer
38 views

Bayesian Linear Regression, trouble with posterior. Variance equal identity

I am trying to solve the following problem. If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
0
votes
0answers
8 views

What is the prior of $\ell_{2,1}$ loss in Multi-Task learning?

We all know Laplacian prior is the prior for Lasso, as the MAP of a Bayesian setting. Multi-task lasso is a generalized lasso for multi-task problems, which encourages group-wise sparisty. However, ...
1
vote
0answers
17 views

analogy on Tenenbaum's phd thesis (on prior, likelihood, posterior)

This is from the book: Machine Learning from a Probabilistic Perspective page 69 and 70. There is a very interesting analogy/explanation on how to visualise Prior, Likelihood and Posterior ...
0
votes
0answers
42 views

Deriving posterior mean with horseshoe prior

I want to decompose a matrix $S \in \mathbb{R}^{D \times D}$ as below $$S=vv^T $$ where $v_i\mid\lambda_i,\tau_i \sim N(0,\lambda^2_i\rho^2_i)$, $\lambda_i \sim Cauchy^+(0,1)$ i.e $v$ has horseshoe ...
2
votes
0answers
25 views

Reproducing a didactic example of Lindley (1993)

Lindley (1993) discusses the following mixed discrete and continuous prior for the tea tasting lady experiment, where $\pi$ is probability of a correct classification: $p(\pi=0.5) = 0.8$ (discrete ...
0
votes
1answer
48 views

Questions about the principles of Bayesian analysis + R [closed]

Let's say I have a data of flywing lengths which is identically distributed (normal). (data: https://seattlecentral.edu/qelp/sets/057/s057.txt). I want to estimate the mean (theta). I have to choose ...
2
votes
2answers
42 views

Borrowing observations for prior probability in Bayesian Inference

For the purposes of Bayesian Inference, is it assumed that the historical observations used for the prior probability values must be from the exact entity for which you are looking to calculate the ...
2
votes
0answers
11 views

Quantifying a reduction in prior uncertainty over several experiments

I am interested in how to quantify reductions in uncertainty about the size of an experimental effect over a series of studies which, for hypothetical reasons, preclude the merging of data. I would ...
2
votes
0answers
28 views

Posterior distribution of Bernoulli distribution

The pdf of X | $\theta$ is given by $\theta^x (1- \theta)^{1-x}$ and its prior distribution is given by $p(\theta) \frac {1} {B(\alpha, \beta)} \theta^{\alpha - 1} (1 - \theta)^{\beta - 1}$ where $...
0
votes
0answers
34 views

Posterior Distribution Decaying to Zero (Underflow Problem)

Say I have a coin. I have no information about this coin; I have no reason believe that it's $p = 0.5$, or anything else, for that matter. So, I use a uniform prior: ...
0
votes
0answers
11 views

Best way to compare observed data points to estimated distribution of expectations

I am interested in comparing individual's expectations to reality (i.e. to see if individuals are optimistic or pessimistic). I have data that allows me to estimate the prior distribution of outcomes ...
0
votes
0answers
12 views

Gibbs Sampling attempt at a simple Coxian distribution

I have the following Coxian model for inter-arival times ($x_i$) that has $C_x^2 < 1$: $$ p(x_i\mid \lambda,\theta) = \theta \lambda^r x_i^r e^{-\lambda x_i} + (1-\theta)\lambda e^{-\lambda x_i} $...
2
votes
0answers
33 views

Help with prior selection

Say I have two samples of univariate data, both of equal size: $X_1, \ldots, X_n \overset{\text{iid}}{\sim} \text{Normal}\left(\mu_1, \sigma^2_1\right)$ and $Y_1, \ldots, Y_n \overset{\text{iid}}{\sim}...
3
votes
3answers
635 views

Why are flat priors said to be proportional to a constant?

I'm a little confused why everyone writes a flat prior as $f(\theta) \propto c$. In this instance couldn't they just write $f(\theta)=c$? A uniform distribution always a has a constant density ...
5
votes
0answers
725 views

Bayes-Poincaré solution to the Behrens-Fisher problem 2: calculations for Jeffreys’ priors [closed]

In a previous post Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?, the classical Bayesian and likelihoodist solutions to 2-sample tests for comparison and the ...
2
votes
1answer
229 views

Understanding how the determinant of the multidimensional normal likelihood can overrule the prior probability

I am doing Bayesian inference. I have a normal prior probability distribution of some theoretical parameter $\theta$ and I am trying to update my knowledge of $\theta$ using some data $D$ and a model $...
2
votes
0answers
33 views

Posterior with a much larger uncertainty than the prior

I have done an MCMC analysis with many variables. One of my nuisance parameters has a Normal prior distribution with mean 0 and standard deviation 1. The posterior distribution for this parameter has ...
1
vote
0answers
25 views

Why do non-informative a priori distributions be chosen to compare the Bayesian and frequentist estimation method?

For example for GARCH models $$\sigma_t^2=\alpha_0 +\alpha_1 y_{t-1}^2 + \beta_1 \sigma^2_{t-1}$$ it is usual to use as distributions for the parameters of truncated normal distributions with very ...
0
votes
1answer
42 views

pymc3: Updating the standard error prior

I am estimating a Bayesian multiple regression using continuous data on both the dependent variable and the regressors. My goal is to iteratively estimate the coefficient distributions as more data ...
1
vote
0answers
54 views

Why do the non-informative a priori distributions give better results than the frequentist estimate?

For example, in the specific case of Markov-Switching GARCH models why is a non-informative prior distribution chosen for GARCH models with Bayesian estimation and why is this approach better than the ...
0
votes
1answer
15 views

What kind of a priori distribution for the Markov Switching models?

Why in the Markov-Switching models is chosen as prior distribution for the probability of the transaction as follows: $$f(P) \propto \prod_{i=1}^K \left(\prod_{j=1}^K p_{i,j}\right) I \left\{0 < ...
1
vote
1answer
43 views

Geometric distribution with a capped number of trials - finding expectation and prior predictive distribution

So I am modeling a random variable which follows a geometric distribution with probability $\theta$ except that the total number of trials is capped at some value $n$. I.e., the probability mass ...
1
vote
3answers
75 views

Bayesian Inference: Feeding Posterior back in as Prior

I've just started reading about Bayesian Inference, and one thing I've wondered about is if it's possible to feed the posterior in as a new prior for a new model, using the same data. Or would that ...
0
votes
0answers
30 views

Modeling the Probability of Falling Asleep

Problem The goal of this exercise is to estimate $p = \text{Pr}(\text{fall asleep})$, the probability of falling asleep imminently, as a mental exercise for those nights when sleep doesn't seem to be ...
1
vote
1answer
69 views

Mechanics for combining likelihood and prior in non-trivial case

Bayes rule is simple enough on its face: $$ pr(B|A) = \frac{pr(A|B)pr(B)}{pr(A)} $$ If these things are known scalar probabilities, the answer is simple to compute. But I'm failing to understand ...
2
votes
0answers
14 views

Using Bayesian formula to apply uncertainty to topic probability given length of document

After computing topics ($z$) over a word network, I'm assigning topic probability to documents, following: $$p(z|d) = \sum_{w_i}p(z|w_i)p(w_i|d)$$ with $d$ being a document composed by $w_{i...n}$ ...
4
votes
1answer
856 views

Bayes-Poincaré solution to k-sample tests for comparison and the Behrens-Fisher problem?

I’d like to share and submit for (dis)approval and discussion yet another, simple but original (to the best of my knowledge) Bayesian solution to the classical problem of comparing k samples or groups,...
2
votes
1answer
59 views

Why does Stan initialize an MCMC chain with a random value generated uniformly from [-2, 2] instead of a random value generated from the prior?

From Stan reference, The default is to randomly generate initial values between -2 and 2 on the unconstrained support It seems to me that it makes more sense to randomly generate initial values ...
0
votes
0answers
32 views

Why do we integrate to obtain prior distributions?

In the test for difference in means with unknown variance, it's stated that in order to obtain prior distributions, we have to take the double integral with respect to mean and to the variance? ...