In Bayesian statistics a prior distribution formalizes information or knowledge (often subjective), available before a sample is seen, in the form of a probability distribution. A distribution with large spread is used when little is known about the parameter(s), while a more narrow prior ...

learn more… | top users | synonyms

2
votes
1answer
33 views

Ratio of Gamma distributed variables with different parameters

I encounter a problem which I thought I can handle, however, I struggle a lot with finding a solution: The following setting applies: I want to compute the posterior probability of an event, which is ...
1
vote
2answers
105 views

Classification: Selecting final label using prior information on class distribution

Using R, let's say that I have the following (dummy) data. ...
1
vote
1answer
19 views

Does class weighting introduce bias in Random Forest classifier?

I want to use a Random Forest classifier to stratify a strongly imbalanced population of samples. During training I used class weighting to weight the vote for each class by considering its ...
0
votes
1answer
22 views

setting log-uniform priors in Stan

I have been using Stan for a couple months now and I want to adopt a log-uniform prior on some parameter array real theta[N]. I want to do something like a ...
3
votes
0answers
37 views

Incorporate an external estimate of probability as predictor in a logistic regression model

I am predicting a binary outcome (e.g., credit default) with logistic regression. For each observation, in addition to my own observed predictors, I have obtained a probability estimate from an ...
1
vote
1answer
59 views

What are the hyperparameters? [duplicate]

I find the meaning of hyperparameters not always clear. The hyperparameters are defined as "the parameters of the prior". Suppose that one has prior information about a certain parameter $\theta$. ...
1
vote
1answer
47 views

Interpretation in histogram of empirical posterior distribution

I'm having trouble to understand the following histograms I know that the posterior distribution in this case is just the empirical cumulative $$P(\rho\leq c)=\frac{1}{n}\sum_{i=1}^n \mathbb{...
10
votes
2answers
197 views

Making a Bayesian prior from a frequentist result

How should one go about turning a frequentist result into a Bayesian prior? Consider the following pretty generic scenario: An experiment was conducted in the past and a result on some parameter $\...
1
vote
0answers
28 views

Bayesian MCMC Fitting

I am doing a Bayesian MCMC fit using emcee in python. I first maximize the log of the likelihood and use the results as initial parameter starting points in my MCMC. I am using a uniform prior and ...
0
votes
0answers
33 views

Nuisance Parameter in Bayesian MCMC

I am doing a Bayesian MCMC fit to some data using a simple model and I want to understand how to handle nuisance parameters. I am looking at this tutorial. The model is a line: $$y = m x + b$$. The ...
1
vote
1answer
15 views

Is it reasonable to have a zero mean Gaussian prior for the coefficients of an AR(p) process, assuming it is stable?

I wanna perform parameter estimation of an underlying AR(p) process given some data. Let's say it's stable. For example an AR(2) process is stable when the conditions $a_2 - a_1 < 1,$ $a_2 + a_1 &...
3
votes
1answer
131 views

Laplace's law of succession using different priors

Laplace's law of succession is a well-known rule, relying on Bayes' theorem. A possible proof of the rule of succession can be found on Wikipedia. Note that for this proof we use a uniform ...
1
vote
1answer
55 views

How to include prior information about target pdf in MCMC

Is there a somewhat principled way to include prior information about a target density $f(x)$ in a sampling (MCMC) algorithm? [This is a much better formulated version of this question, which I am ...
0
votes
1answer
43 views

Conjugate prior for parameter W when the likelihood is normal with mean and variance both functions of W

Suppose that $x$ is an observable scalar variable and $$ x \thicksim N(W\mu_0,W^2\sigma_0^2) $$ Where $W$ is a parameter that must be estimated from data, and $\mu_0$ , $\sigma_0$ are known constants....
7
votes
0answers
100 views

About the Jeffreys prior for multivariate model

In certain cases, the Jeffreys prior for a full multidimensional model is clearly inadequate, this is for example the case in: $$ y_i=\mu + \epsilon_i $$ where $\epsilon \sim N(0,\sigma^2)$, $\mu$ and ...
0
votes
0answers
9 views

Standard notation to indicate certain distributions

On a figure that I wish to annotate to indicate what prior distributions were used in an analysis, I need a shorter way of indicating a $\text{Cauchy}(0, \sigma)$ distribution and saving just 3 or 4 ...
15
votes
2answers
245 views

What is the relation behind Jeffreys Priors and a variance stabilizing transformation?

I was reading about the Jeffreys prior on wikipedia: Jeffreys Prior and saw that after each example, it describes how a variance-stabilizing transformation turns the Jeffreys prior into a uniform ...
4
votes
2answers
171 views

Does “improper” posterior or prior refer to a density function that does not integrate to 1 or to one that does not integrate to a finite value?

I am a bit confused about improper priors and posteriors. I have seen references that classify a prior or posterior probability density function as "improper" if the integral over infinite support ...
1
vote
0answers
60 views

Inverse gamma posterior

I am working on bayesian analysis and I have a normal likelihood function and an inverse gamma (IG) prior for the parameter $\lambda$. I have the following posterior: $$ \propto \frac{1}{\lambda^{\...
2
votes
1answer
83 views

Bayesian derivation of unbiased maximum likelihood estimator

I was recently reading an old NIPS paper by Bishop and Qazaz where they claim that an unbiased estimator for variance, based on $N$ Gaussian $\rm i.i.d.$ samples with unknown mean and unknown variance,...
2
votes
3answers
96 views

Verifying and/or changing priors assumptions on Bayesian ANOVA

I am performing a Bayesian analysis of around 1500 data, divided into 2 factors, one that I am interested x1, and the id-variable for the paired/within-subject x2. x1 has 15 levels, and x2 around 100 ...
1
vote
0answers
27 views

choosing prior parameters for variational mixture of Gaussians

I am implementing a vanilla variational mixture of multivariate Gaussians, as per Chapter 10 of Pattern Recognition and Machine Learning (Bishop, 2007). The Bayesian approach requires to specify (...
3
votes
1answer
97 views

Probability distribution transformation of variables question

Problem: Hi there, I'm stuck trying to derive an equation stated in a research paper relating to Bayesian statistics in Cosmology (the paper is: http://mnras.oxfordjournals.org/content/398/4/2049....
0
votes
0answers
28 views

Implication of using independent priors for means of joint normally distributed random variables

I am using Bayesian methodology to estimate parameters of joint distribution(Multivariate normal) of random variables $(y_1, y_2) \sim N(\mu, \Sigma)$. I implemented the code for finding the posterior ...
0
votes
0answers
28 views

How to find the posterior distribution and posterior mode of Beta given an exponential prior distribution and binomial data?

This is the question I'm working on: I have already completed part (a). For part (b) i, I modeled y1 given Beta and n1 as Binomial(n1, p1 = d1/(d1+Beta)). For part (b) ii, I showed that the ...
1
vote
0answers
22 views

Comparing the rates of Poisson distribution using Bayesian inference

In the 'Theory of Probability' book by Sir Harold Jeffreys, (5.15), the form of the Bayes Factors that he derives for the comparison of Poisson rates is the same as that of Binomial rates. But I did ...
2
votes
1answer
36 views

Risk and posterior expectation Bayesian Statistics

Consider $x\sim B(n,\theta)$ with $n$ known a)If $\pi(\theta)\sim Beta(\sqrt{n}/2,\sqrt{n}/2)$ give the associated posterior distribution and posterior expectation $\delta^\pi(x)$ b)Show ...
1
vote
2answers
70 views

Why is computing the Bayesian Evidence difficult?

In Bayesian estimation, we need to compute the normalizing factor P(X). Say that our parameter space was y. Then in order to compute the Bayesian evidence we'd need ...
0
votes
0answers
14 views

Incorporating prior class probability in decision trees

How do we incorporate prior class distributions in algorithms such as CART? I read that it would have an impact on the splitting of the tree (if the distribution is different than what we have in the ...
4
votes
0answers
25 views

Examples of usage of community of priors or why aren't they used more commonly?

Kass and Greenhouse (1989) proposed using "community of priors" (see also Fayers et al, 1997; 2000). As described by Spiegelhalter (2004), they can be seen as a range of viewpoints that should be ...
0
votes
0answers
12 views

Prior estimation for Dirichlet Process Clustering

I wrote this code for Dirichlet Process Clustering using Chinese Restaurant Process in which a parameter ...
3
votes
1answer
58 views

Prior for Bayesian multiple logistic regression

I was wondering how I could incorporate a prior to form a posterior distribution for multiple logistic regression. More specifically, I am working with basketball data, where the response variable is ...
0
votes
1answer
36 views

Dirichlet Process Clustering Prior

I'm doing dirichlet process clustering where dirichlet priors are used as: with CRP representation as: First customer will always choose first table. Second will choose already occupied table with ...
8
votes
2answers
358 views

Do Bayesian priors become irrelevant with large sample size?

When performing Bayesian inference, we operate by maximizing our likelihood function in combination with the priors we have about the parameters. Because the log-likelihood is more convenient, we ...
4
votes
0answers
45 views

Priors on variable ordering and/or percentile ranking

Consider a set of variables $\mathbf{X}$ = $X_1 \ldots X_n$ where each variable is $\in [0,1]$. I am modeling an inference problem on these variables. Among other things, I have the following prior ...
3
votes
2answers
87 views

Incorporating Prior Class Probability Distribution in Logistic Regression

I am amazed that I can not find any articles / lectures about how one can incorporate Prior Class Probability Distributions in classifiers like Logistic Regression or Random Forest. So my question ...
3
votes
0answers
74 views

Horseshoe priors and random slope/intercept regressions

I'm interested in using the horseshoe prior (or the related hierarchical-shrinkage family of priors) for regression coefficients of a traditional multilevel regression (e.g., random slopes/intercepts)....
0
votes
0answers
25 views

conjugate prior for exponential distribution

If there is an exponential distribution $$p(x | \theta) = \theta\,e^{-x\theta}\mathbb{I}_{x>0}\, ,$$ what is a good conjugate prior? Also, will the posterior mean is a convex combination of prior ...
1
vote
1answer
40 views

Prior on Precision or variance

I've been reading this tutorial on variational bayes which talks about sparse Bayesian learning (Relevance Vector Machines if you prefer). In the paper they put a Gamma prior on the precision ...
8
votes
3answers
256 views

Gaussian likelihood + which prior = Gaussian Marginal?

Given a Gaussian likelihood for a sample $y$ like $$p(y|\theta) = \mathcal{N}(y;\mu(\theta),\Sigma(\theta))$$ with $\Theta$ being the parameter space and $\mu(\theta)$, $\Sigma(\theta)$ arbitrary ...
0
votes
0answers
20 views

Ising-Like Priors with Fractal Boundaries (Application to Image Processing)

Overview: I'm interested in looking for priors that "look a little like" the Ising model, but have different large-scale behaviour. In particular, I'm looking for priors that give rise to large ...
0
votes
1answer
32 views

Posterior from a Poisson likelihood and prior

I have the following Poisson mass function: $$p(y| \theta) = \frac{\theta^y e^{\theta}}{y!} $$ Which has a corresponding likelihood for n independent realizations of y as follows: $$\frac{e^{-n\...
1
vote
1answer
81 views

How to fit a Pareto distribution via Bayesian estimation (with a Pareto prior)?

I don't know Bayesian statistics very well, so I don't know if the question makes sense. Let me give an example. We assume that the income distribution of a country is a Pareto distribution (the ...
1
vote
0answers
88 views

How Do I choose parameters of prior on regression coefficients in a Bayesian linear model?

I'm trying to perform a linear regression in a Bayesian way. The response is normal,the prior I would like to put over Beta (vector of regression coefficients) and Sigma^2 (variance of the error ...
2
votes
2answers
110 views

priors for Gamma shape and scale parameters

I have a random variable $X$ that is Gamma distributed with unknown parameters $\alpha$ and $\beta$: $$ X\sim \text{Gamma}(\alpha, \beta) $$ I now want to estimate $\alpha$ and $\beta$ from samples $...
2
votes
1answer
148 views

How to determine posterior distribution of the parameter in a binomial

Assuming that I performed n iid tests, and the total number of test is n which is a fixed value, and the observaton of 1 which corresponding to successful results is X observations yeild with ...
2
votes
0answers
70 views

Is Independent jeffreys prior different from independent reference prior?

I have a model involving two scalar parameters $\theta_1$ and $\theta_2$ and derived the Jeffreys prior for $\theta_1$ and $\theta_2$ independently (so for, e.g. $\pi(\theta_1)$, setting in the ...
2
votes
0answers
55 views

How to write unnormalized posterior when prior is a mixture of continuous and discrete

Suppose I want to do bayesian inference on the regression problem $\beta$ for Y = X$\beta$ + $\epsilon$ for $\epsilon_i \sim N(0,\sigma^2)$. The complication is that the prior for each component $\...
1
vote
1answer
50 views

Updating Bayesian prior & likelihood for A/B test

I'm fairly new to bayesian. I'm trying to edit a bayesian python code for $A/B$ test analysis. I'm using uninformative priors as a beta distribution, so my $\alpha$ & $\beta$ parameters are $1$ &...
6
votes
1answer
57 views

Why does probabilistic PCA use Gaussian prior over latent variables?

I am currently reading papers about probabilistic PCA and I am wondering why is Gaussian prior (and not some other prior) chosen for the latent variables? Is it just because it's simple or is there ...