Questions tagged [empirical-bayes]

Procedures for statistical inference in which the prior distribution is estimated from the data.

Filter by
Sorted by
Tagged with
22
votes
1answer
1k views

How is empirical Bayes valid?

So I just finished reading a great book Introduction to Empirical Bayes. I thought that the book was great, but building priors from the data felt wrong. I was trained that you come up with an ...
20
votes
2answers
3k views

Cross-validation vs empirical Bayes for estimating hyperparameters

Given a hierarchical model $p(x|\phi,\theta)$, I want a two stage process to fit the model. First, fix a handful of hyperparameters $\theta$, and then do Bayesian inference on the rest of the ...
12
votes
1answer
2k views

hierarchical Bayesian models vs. empirical Bayes

Would you consider the HBM vs EB to be two alternatives in which the hyperparameters are "in the game" of being sampled/estimated/etc.? There is clearly a connection between these two. Would you ...
12
votes
1answer
2k views

Is there a connection between empirical Bayes and random effects?

I recently happened to read about empirical Bayes (Casella, 1985, An introduction to empirical Bayes data analysis) and it looked a lot like random effects model; in that both have estimates shrunken ...
10
votes
0answers
188 views

Empirical Bayes (In)Admissibility

Sticking to a pure Bayesian approach to statistics with proper priors most of the time leads to admissible estimators. Nevertheless there is good reason to use Empirical Bayes in many cases, and the ...
6
votes
1answer
185 views

Using empirical Bayesian estimation (Gamma-Poisson) to analyze high arrival counts (n ~= 5000)

Here's a problem I'm currently working on, as well as the empirical Bayesian approach I'm using. I'd like to make sure my approach is grounded in solid statistical theory. I have a set of entities $e=...
6
votes
1answer
187 views

Bootstrapping the data to set up a prior

I am using a Gaussian model with a conjugate Normal-Inverse-Wishart (NIW) prior, as described here. The advantage of this approach is that the marginal likelihood $p(y)$, which is what I am interested ...
5
votes
1answer
118 views

Can “cross-validation” be used to choose a prior?

To be clear, I doubt I am using the term "cross-validation" correctly here; what I am suggesting also seems similar to "boot-strapping" and "hyperparameter tuning". Terminology is not my strength. ...
4
votes
1answer
546 views

Hyper-parameter estimation for Beta-Binomial Empirical Bayes

I am reading a paper Illustrating empirical Bayes methods and in the paper the author uses method of moments to derive the value of an estimate. In equation 17 the author gives the following marginal ...
4
votes
1answer
119 views

Expectation of inverse chi-square random variable (Efron, 2010)

Let $z_i|\mu\sim N(\mu,1)$ and $\mu\sim N(B,A)$ for $i=1,\dots,N$, the implication is that $z_i|B\sim N(B,A+1)$. Define $S=\|\textbf{z}\|^2$, and let $B=0$, then $S\sim(A+1)\chi^2_N$ since $\|\...
4
votes
1answer
306 views

Empirical Bayes vs “non-informative” priors

I am familiar with the mechanics with both methods, but don't know what factors I should consider when choosing between these two approaches for adjusting a prior. I would imagine that, on a case by ...
3
votes
1answer
305 views

Empirical Bayes/MCMC references

I'm interested in references for running empirical Bayes (EB) in conjunction with MCMC. The closest thing I've found to what I'm looking at is a surprisingly recent and somewhat obscure paper ...
2
votes
1answer
32 views

How do Bayesian hierarchical models adaptively learn the prior?

It seems the main difference between a hierarchical and a non hierarchical model is that the hierarchical model learns the prior. That is it adaptively comes up with a regularizing prior to be applied ...
2
votes
1answer
160 views

Learning prior distribution from data

Suppose I have a dataset. How can I learn the prior distributions of the parameters of a model from this data? I want to learn the prior from this data in order to use them in a Bayesian model. Sorry ...
2
votes
1answer
214 views

Empirical Bayes: method of moments

The model for the data is $X_{i}$ ~ $Bin(n_{i},\theta_{i})$ (iid, $i=1,...,k$). The prior distribution is $\theta_{i}$ ~ $Beta(\alpha,\beta)$. How do we choose (and deduct) moment estimators for $\...
2
votes
1answer
75 views

James-Stein Estimator with unequal numbers in groups

In the book Computer Age Statistical Inference the James-Stein estimator is introduced. Brad Efron runs through an example where batting averages are estimated from each players 90 at-bats. $$p_i\sim ...
1
vote
1answer
36 views

Poisson-gamma posterior mean expectation

Let's have a gamma prior $\lambda\sim \operatorname{Gamma}(a,b)$ (mean: $\frac{a}{b}$) With Poisson data $Y\mid \lambda\sim \operatorname{Pois}(N\lambda)$ (mean: $N\lambda$) The posterior is $\...
1
vote
1answer
87 views

Empirical Bayes Estimation

Suppose $X_i$ conditioned on $\mu$ is iid $N(\mu, \sigma^2)$ and $\mu$ is distributed as $N(\mu_0, \tau^2)$. Is there a way to estimate $\tau^2$?
1
vote
1answer
1k views

What is the empirical Bayes estimator for a gamma-Poisson with more than 1 observation for each Poisson parameter?

I am looking at the Wikipedia entry for empirical Bayes, but it's a bit confusing - it seems to me the solution must apply only to the case in which there's only $n=1$ sample $y$ for each $\theta$ and ...
1
vote
1answer
636 views

How to pick starting parameters for MASS:fitdist() with the beta distribution?

I have data set of ~700k yes/no events that I want to first aggregate on various features (e.g. group by average), always resulting in a 34 length vector. From there, I want to fit a beta ...
1
vote
1answer
52 views

Robbins estimate Empirical Bayes

From the compound sampling model where: $Y_i | \theta_i \sim Poi(\theta_i)$ The marginal distribution of $\theta_i$ is $G$, non-parametric. We get that the Bayes estimate of $\theta_i$ under ...
1
vote
0answers
19 views

Is expectation maximization an example of empirical Bayes?

I don't think I truly understand what methods are classified as "empirical Bayes". Is expectation maximization considered an example of this?
1
vote
0answers
41 views

Why the local Bayes fdr is greater than the Bayes FDR?

My question is related to empirical Bayes and large-scale inference. It is explained that the local Bayes false discovery rate (fdr) is greater than the Bayes false discovery rate (FDR). It is argued ...
1
vote
0answers
321 views

How to calculate the inverse cdf for a sample with unknown distribution?

I'm stuck with an exercise that I'll just write down first: Based on a sample $x_1,...,x_{47}$, that can be considered coming from an unknown distribution, we study a qq-plot where the empirical ...
0
votes
1answer
22 views

Empirial Bayes for means

I have data for US ~3100 counties where the variable is a mean score based on a sample. However, for many small counties, the sample size is quite small (like 5), and so these mean values fluctuate a ...
0
votes
0answers
26 views

Modeling the Probability of Falling Asleep

Problem The goal of this exercise is to estimate $p = \text{Pr}(\text{fall asleep})$, the probability of falling asleep imminently, as a mental exercise for those nights when sleep doesn't seem to be ...
0
votes
0answers
32 views

Get posterior distribution of categorical variable given empirical continuous-categorical priors?

Suppose I have categorical variable $Z \in D$ defined for some finite domain $D$. I also have a continuous variable $X \in \mathbb{R}$ which is observed. From historical data samples I have the ...
0
votes
0answers
64 views

question about empirical Bayes(EB)

In David Robinson's blog:Understanding empirical Bayes estimation (using baseball statistics) he used the hit ratio to fit the beta distribution as a prior distribution,Where hit rate = hits/total. ...
0
votes
0answers
137 views

Why does Empirical Bayes work in my simple case?

I have a problem where I am trying to classify data into two groups using a single parameter. The distribution of this parameter is Gaussian for two groups, so what I'm dealing with is two overlapping ...