Questions tagged [prior]

In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior distribution represents a greater degree of information.

Filter by
Sorted by
Tagged with
5
votes
3answers
632 views

What methods can be used to specify priors from data?

Background I am generally interested in learning appropriate methods of using data to specify priors. A previous question asks how to elicit priors from experts and received some good recommendations....
4
votes
1answer
975 views

Interpreting prior and posterior

I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by ...
6
votes
2answers
2k views

Definition of improper priors

If a prior integrates to a finite constant that is not 1, is it still considered proper? Is a prior only improper if it integrates to an nonfinite value?
5
votes
2answers
309 views

When a prior distribution would not be overwhelmed by data, regardless of the sample size?

I came across a question 8 at the end of chapter 3 of the book: "Give two simple examples showing a case in which a prior distribution would not be overwhelmed by data, regardless of the sample size"...
4
votes
1answer
2k views

Ratio of Gamma distributed variables with different parameters

I encounter a problem which I thought I can handle, however, I struggle a lot with finding a solution: The following setting applies: I want to compute the posterior probability of an event, which is ...
3
votes
1answer
178 views

Estimating probability or frequency with low N?

I am trying to estimate the probability of an event using a low number of observations. The naive estimator $\hat{p} =\frac{\text{number of positive observations}}{\text{total number of observations}}...
6
votes
1answer
554 views

The unit information prior and its BIC approximation

Just moments ago, I asked this question because I've been reading Wagenmakers 2007. I now have a better understanding of what a unit information prior is and can push my knowledge further with two (by ...
5
votes
1answer
74 views

Numbers of draws on a modified Bernouilli process

Here is the setup: Bob runs an experiment: he flips a coin N times (between 0 and +$\infty$). The coin has a probability p of landing on heads. Bob starts with zero points. For each head, Bob scores a ...
5
votes
1answer
3k views

Understanding definition of informative and uninformative prior distribution

When using the "non-informative" prior $\pi(\mu,\sigma)\propto\frac{1}{\sigma^2}$ where $\pi(\mu)\propto1$ and $\pi(\sigma^2)\propto\frac{1}{\sigma^2}$ Where is the no information for the ...
0
votes
1answer
244 views

Include prior knowledge in regression model

I've a classical dataset with real attributes and I want to perform a regression. But, not all the entries in the training dataset are trustworthy; there is an attribute that I can turn into a ...
0
votes
1answer
129 views

Is this posterior probability integral right?

From Wiki: where , k is binomially distributed, and I'm not sure about u. I'm thinking that the second line should be: I mean, if we let X represent the toss of a die, then $P(X = 1, 2, 3, 4, 5)...
32
votes
1answer
20k views

What are the properties of a half Cauchy distribution?

I am currently working on a problem, where I need to develop a Markov chain Monte Carlo (MCMC) algorithm for a state space model. To be able to solve the problem, I have been given the following ...
22
votes
6answers
5k views

Posterior very different to prior and likelihood

If the prior and the likelihood are very different from each other, then sometimes a situation occurs where the posterior is similar to neither of them. See for example this picture, which uses normal ...
23
votes
2answers
2k views

What is/are the implicit priors in frequentist statistics?

I've heard the notion that Jaynes claims frequentists operate with an "implicit prior". What is or are these implicit priors? Does this mean frequentist models are all special cases of Bayesian ...
18
votes
4answers
2k views

How do Bayesian Statistics handle the absence of priors?

This question was inspired by two recent interactions I had, one here in CV, the other over at economics.se. There, I had posted an answer to the well-known "Envelope Paradox" (mind you, not as the "...
12
votes
1answer
520 views

When should I be worried about the Jeffreys-Lindley paradox in Bayesian model choice?

I am considering a large (but finite) space of models of varying complexity which I explore using RJMCMC. The prior on the parameter vector for each model is fairly informative. In what cases (if any)...
10
votes
1answer
2k views

Softmax regression bias and prior probabilities for unequal classes

I'm using Softmax regression for a multi-class classification problem. I don't have equal prior probabilities for each of the classes. I know from Logistic Regression (softmax regression with 2 ...
8
votes
2answers
9k views

How is the inverse gamma distribution related to $n$ and $\sigma$?

Given that the posterior estimate of $\sigma'^{2}$of a normal likelihood and an inverse gamma prior on $\sigma^2$ is: $$\sigma'^{2}\sim\textrm{IG}\left(\alpha + \frac{n}{2}, \beta +\frac{\sum_{i=1}^n{...
8
votes
1answer
545 views

Model with admissible estimator(s) that are not the Bayes estimator for any choice of prior?

Every Bayes estimator is admissible, to the best of my knowledge. (Related questions - 1,2.) I recall my professor mentioning once during a lecture that, at least as rough intuition, the converse is ...
5
votes
2answers
2k views

Why is computing the Bayesian Evidence difficult?

In Bayesian estimation, we need to compute the normalizing factor P(X). Say that our parameter space was y. Then in order to compute the Bayesian evidence we'd need ...
10
votes
1answer
4k views

Bayes factors with improper priors

I have a question regarding model comparison using Bayes factors. In many cases, statisticians are interested on using a Bayesian approach with improper priors (for example some Jeffreys priors and ...
9
votes
3answers
451 views

Does a Bayes estimator require that the true parameter is a possible variate of the prior?

This might be a bit of a philosophical question, but here we go: In decision theory, the risk of a Bayes estimator $\hat\theta(x)$ for $\theta\in\Theta$ is defined with respect to a prior distribution ...
7
votes
1answer
7k views

What is the difference between "priors" and "likelihood"?

I have been reading some papers regarding statistics and I seem to be confusing the terms priors and likelihood. Would it be possible to explain the difference between the two terms? I am interested ...
7
votes
1answer
6k views

Choosing prior for $\sigma^2$ in the normal (polynomial) regression model $Y_i | \mu, \sigma^2 \sim \mathcal{N}(\mu_i, \sigma^2)$

I have the polynomial regression model $$Y_i | \mu, \sigma^2 \sim \mathcal{N}(\mu_i, \sigma^2), i = 1, \dots, n \ \text{independent}$$ $$\mu_i = \alpha + \beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x^2_{...
18
votes
2answers
1k views

What is the relation behind Jeffreys Priors and a variance stabilizing transformation?

I was reading about the Jeffreys prior on wikipedia: Jeffreys Prior and saw that after each example, it describes how a variance-stabilizing transformation turns the Jeffreys prior into a uniform ...
16
votes
3answers
16k views

Jeffreys Prior for normal distribution with unknown mean and variance

I am reading up on prior distributions and I calculated Jeffreys prior for a sample of normally distributed random variables with unknown mean and unknown variance. According to my calculations, the ...
10
votes
2answers
1k views

Allow data to dictate the priors and then run the model using these priors? (e.g., data-driven priors from same data set)

It is my understanding that we should not be allowing the same data set we are analyzing to drive/define what the prior distributions look like in a Bayesian analysis. Specifically, it is ...
8
votes
1answer
872 views

Choosing non-informative priors

I am working on a model relying on an ugly parametrized function acting as a calibration function on a part of the model. Using a Bayesian setting, I need to get non-informative priors for the ...
6
votes
1answer
3k views

How do I combine multiple prior components and a likelihood?

Lets imagine I am comparing two groups of animals (treatment/control). There is previous data from cell cultures indicating the treatment should have a positive effect. This gives me "prior component ...
4
votes
2answers
3k views

Use of prior and posterior predictive distributions?

I understand the prior and posterior distributions and I have read what the prior and posterior predictive distributions are. However, I don't really see the point of knowing them. Knowing more ...
4
votes
0answers
244 views

Intuition regarding Bayesian pseudo-priors

One approach to model comparison in a Bayesian framework uses a Bernoulli indicator variable to indicate which of two models is likely to be the "true model". When applying MCMC-based tools for ...
10
votes
1answer
2k views

Hyperprior distributions for the parameters (scale matrix and degrees of freedom) of a wishart prior to an inverse covariance matrix

I'm estimating several inverse covariance matrices of a set of measurements across different subpopulations using an wishart prior in jags/rjags/R. Instead of specifying a scale matrix and degrees ...
9
votes
2answers
3k views

Incorporating Prior Class Probability Distribution in Logistic Regression

I am amazed that I can not find any articles / lectures about how one can incorporate Prior Class Probability Distributions in classifiers like Logistic Regression or Random Forest. So my question ...
8
votes
1answer
858 views

Why not use Beta(1,1) as boundary avoiding prior on a transformed correlation parameter?

In Bayesian Data Analysis, chapter 13, page 317, second full paragraph, in the modal and distributional approximations, Gelman et al. write: If the plan is to summarize inference by the posterior ...
7
votes
0answers
3k views

Hyper-prior for negative binomial in hierarchical model using JAGS/BUGS

Below I'm using a negative binomial because it is more flexible than a simple poisson model. The data are counts $y$ of events for 16 individuals $x$. There are 14 counts (i.e. counting periods) for ...
7
votes
1answer
3k views

MAP estimation as regularisation of MLE

Going through the Wikipedia article on Maximum a posteriori estimation, it got confusing after reading this: It is closely related to the method of maximum likelihood (ML) estimation, but employs ...
4
votes
2answers
1k views

Shrinkage priors

I am building a Bayesian model where I to put shrinkage priors such as spike and slab, horseshoe prior, etc on some parameters for feature selection, but I am not able to decide which one is the best. ...
4
votes
1answer
751 views

Gaussian Processes, Prior function of constant, Linear and Polynomial Kernel

In my university course slide, regarding the exponential and Gaussian kernel there are these two relevant pictures. They show the prior function of each kernel: You can see that the resulting prior ...
3
votes
2answers
16k views

Uniform vs Beta(1,1) prior

Is there any difference in applying a uniform prior or a Beta(1,1) prior for your Bayesian analysis ?In which conditions is one preferred over the other ?
4
votes
1answer
2k views

Posterior of the linear regression model with g-Prior

Assume a linear model of the form $$Y=X\beta+\epsilon$$ where $\epsilon$ has a multivariate Normal distribution with mean $0_N$ and covariance matrix $\sigma^2 I_N$. I would like to perform simple of ...
4
votes
1answer
81 views

Can the posterior mean always be expressed as a weighted sum of the maximum likelihood estimate and the prior mean?

See this question. Is this always true? Can the posterior mean always be expressed as a weighted sum of the maximum likelihood estimate and the prior mean (after choosing some appropriate prior)?
2
votes
1answer
348 views

Difference between random effect and fixed effect with regularization/prior

Let's say I have a random effect intercept. For example: lme4::lmer(yield ~ 1 + (1|Batch)) How is that different than just ordinary regression using ...
2
votes
1answer
476 views

Risk and posterior expectation Bayesian Statistics

Consider $x\sim B(n,\theta)$ with $n$ known a)If $\pi(\theta)\sim Beta(\sqrt{n}/2,\sqrt{n}/2)$ give the associated posterior distribution and posterior expectation $\delta^\pi(x)$ b)Show ...
2
votes
1answer
2k views

Conjugate priors for Gamma distribution of unknown $\alpha$ and $\beta$

Per the Wikipedia on conjugate priors link, the conjugate prior for a Gamma of unknown $\alpha$ and $\beta$ is proportional to an expression involving both $\alpha$ and $\beta$ as well as a $\Gamma$ ...
1
vote
1answer
318 views

Why Does the $\propto$ Symbol Replace the $=$ Symbol When Using Bayes' Rule to Convert Posterior Density to Unnormalised Posterior Density?

My textbook says the following: In order to make probability statements about $\theta$ given $y$, we must begin with a model providing a joint probability distribution for $\theta$ and $y$. The ...
9
votes
1answer
4k views

Jeffreys prior for geometric distribution?

What is the Jeffreys prior for the geometric distribution?
8
votes
3answers
1k views

Gaussian likelihood + which prior = Gaussian Marginal?

Given a Gaussian likelihood for a sample $y$ like $$p(y|\theta) = \mathcal{N}(y;\mu(\theta),\Sigma(\theta))$$ with $\Theta$ being the parameter space and $\mu(\theta)$, $\Sigma(\theta)$ arbitrary ...
6
votes
1answer
5k views

half-cauchy prior for scale parameter

I am looking for a prior for a scale parameter for which I have prior knowledge such that: "$\sigma$ typically does not exceed 100." ("typically" meaning that occasionnally this can happen). In such ...
6
votes
3answers
536 views

What happens if I change the range of a flat prior for Bayesian inference?

I am working through an example on doing Bayesian inference on binomial distribution using a flat prior, and trying to understand the impact of choosing a prior. I know that if we work with a flat ...
5
votes
1answer
120 views

Examples of usage of community of priors or why aren't they used more commonly?

Kass and Greenhouse (1989) proposed using "community of priors" (see also Fayers et al, 1997; 2000). As described by Spiegelhalter (2004), they can be seen as a range of viewpoints that should be ...