Questions tagged [empirical-bayes]

Procedures for statistical inference in which the prior distribution is estimated from the data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
34 views

How to combine priors for empirical bayes

I am trying to estimate $Z_{i} = P(Y=1 | A=a_{i}, B = b_{i}, C =c_{i})$ using something like empirical Bayes as in http://varianceexplained.org/r/empirical_bayes_baseball/ aggregating a massive amount ...
user avatar
  • 7,650
1 vote
0 answers
27 views

Which marginal likelihood is maximized in empirical Bayes?

I am reading up on empirical Bayes, and I am finding it really hard to understand what exactly is going on. As far as I understand, it is based on maximizing the marginal likelihood of the data (so ...
user avatar
  • 11
4 votes
0 answers
49 views

Why Are Empirical Bayes Methods Not Considered "Controversial"? [duplicate]

I was reading about Empirical Bayesian Methods and came across the following: My Question: As this text explains, I have often heard that the priors used in Bayesian Methods should be decided prior ...
user avatar
  • 5,650
1 vote
0 answers
58 views

Empirical bayes estimation of variograms for multiple variables

I am looking to estimate variograms, or spatial correlation matrices on all columns of a data matrix $\mathbf{X}_{n\times p}$ with $p>n$. The matrix of spatial coordinates $\mathbf{Y}_{n\times 2}$ ...
user avatar
  • 1,532
1 vote
0 answers
25 views

Empirical Bayes prediction

I've been working with a GLMM in SAS. Using the proc GLIMMIX procedure, I've extracted the EB estimates of the random-effects. I was looking into empirical Bayes estimates and their predictive power. ...
user avatar
  • 23
0 votes
0 answers
16 views

False Discovery Rate From Posterior Samples of Bayesian Models

This is likely a naive question but here goes. There are a number of papers on Bayesian FDR (Efron/Efron and Tibshirani) etc. All of these models seem to assume that the score(p value or z values, or ...
user avatar
0 votes
1 answer
30 views

How to manually adjust predicted probabilities from lm based on prior lognormal distribution parameters?

@drob showed a great example of adjusting batting averages using a beta-based prior distribution. He used a prior calculated Beta distribution to adjust batting averages individually, and it’s as ...
user avatar
2 votes
1 answer
95 views

Is empirical Bayes estimation still possible without common parameters?

In An Introduction to Empirical Bayes Data Analysis by George Casella (1985), it is given that \begin{align} X|\theta &\sim N(\theta,\sigma^2) \\ \theta &\sim N(\mu,\tau^2) \\ \theta|X &\...
user avatar
  • 2,906
3 votes
1 answer
107 views

Is empirical Bayes an iterative scheme?

From the Wikipedia article on the empirical Bayes method (emphasis mine): In, for example, a two-stage hierarchical Bayes model, observed data $y=\{y_{1},y_{2},\dots ,y_{n}\}$ are assumed to be ...
user avatar
  • 2,906
0 votes
0 answers
21 views

Impact of # of Level 1 units on Level 3 Empirical Bayes Estimates

I'm in the midst of fitting a multi-level model using longitudinal data (specifically, working with the Current Population Survey). The model consists of monthly survey waves nested in individuals ...
user avatar
1 vote
0 answers
26 views

Nonparameteric Empirical Estimator For Stochastic Process

Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
user avatar
1 vote
0 answers
39 views

Limiting distribution of iterative applications of Bayes' rule

The question Suppose we iteratively use the posterior as the prior on the same data.* What is the limiting distribution of the posterior? Let's make that precise. The data $X$ and the likelihood ...
user avatar
  • 476
0 votes
1 answer
405 views

How to estimate beta distribution parameters using a beta binomial with empirical bayes

I would like to estimate parameters for a beta distribution using a maximum likelihood approach in python (as mentioned here). I can do this for a beta: ...
user avatar
2 votes
0 answers
41 views

Empirical bayes approach to hypothesis testing

Empirical Bayes seems to be quite successful in point estimation. A classical example would be the Robbins' formula to estimate the Poisson rate. Robbins also published a paper about the application ...
user avatar
  • 88
0 votes
0 answers
56 views

Generating Poisson-Gamma data for empirical parameter estimation in r

I'm currently trying to learn parameter estimation and want to illustrate and compare different credibility methods (limited fluctuation, Bühlmann, Bühlmann-Straub) using both Bayesian and Empirical ...
user avatar
  • 41
4 votes
1 answer
270 views

Using the MLE to select the prior distribution...empirical Bayes?

It was requested that I read the following article for work: https://support.sas.com/resources/papers/proceedings15/1400-2015.pdf In Case II, the author starts by doing two things: First, he computes ...
user avatar
1 vote
0 answers
34 views

Empirical Bayes Estimation: Different transformation, fully different results

I would be grateful for some advice regarding the following problems: As I reviewed some empirical papers, I learned that most of them use a relatively small number of observations for performing ...
user avatar
2 votes
1 answer
912 views

Beta-Binomial regression or Poisson-Gamma model to account for uncertainty in (empricial Bayesian) prior? Explained in simple terms?

I have a dataset of $m$ individuals. For each individual $m$ I have $n_m$ (binomial ) observations with $s_m$ corresponding to the number of 'successes'. I use this data to fit a beta-binomial ...
user avatar
5 votes
1 answer
246 views

Admissible Empirical Bayes Examples

I would like to hear about a few simple empirical bayes estimators that are admissible for high (i.e. at least 3) dimensional parameter space. What are some textbook lollipop examples to study for ...
user avatar
4 votes
2 answers
125 views

Accounting for uncertain information (few observations) in a prior (empirial Bayes)

I did not really know how to choose an adequate title for this question, so please feel free to change it. I have a weird case wherein frequentist and Bayesian philosophies come together. I am ...
user avatar
5 votes
1 answer
179 views

Likelihood of linear mixed effects model

Consider the following model $$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right....
user avatar
  • 873
2 votes
0 answers
79 views

Replicating a Tweedie corrected experiment from Computer Age Statistical Inference

I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411. In this experiment 100 datasets are populated normal random ...
user avatar
  • 943
5 votes
2 answers
272 views

Is appropriate to use empirical Bayes (EB) in this way?

Background. I have data from a study where participants make a series of judgments (a series of decisions with a binomial outcome, either $y=1$ or $y=0$). I have a model of the underlying decision-...
user avatar
  • 2,741
1 vote
1 answer
180 views

Poisson-gamma posterior mean expectation

Let's have a gamma prior $\lambda\sim \operatorname{Gamma}(a,b)$ (mean: $\frac{a}{b}$) With Poisson data $Y\mid \lambda\sim \operatorname{Pois}(N\lambda)$ (mean: $N\lambda$) The posterior is $\...
user avatar
1 vote
1 answer
87 views

Empirial Bayes for means

I have data for US ~3100 counties where the variable is a mean score based on a sample. However, for many small counties, the sample size is quite small (like 5), and so these mean values fluctuate a ...
user avatar
1 vote
0 answers
89 views

Is expectation maximization an example of empirical Bayes?

I don't think I truly understand what methods are classified as "empirical Bayes". Is expectation maximization considered an example of this?
user avatar
2 votes
1 answer
98 views

How do Bayesian hierarchical models adaptively learn the prior?

It seems the main difference between a hierarchical and a non hierarchical model is that the hierarchical model learns the prior. That is it adaptively comes up with a regularizing prior to be applied ...
user avatar
2 votes
1 answer
1k views

Learning prior distribution from data

Suppose I have a dataset. How can I learn the prior distributions of the parameters of a model from this data? I want to learn the prior from this data in order to use them in a Bayesian model. Sorry ...
user avatar
  • 31
0 votes
0 answers
118 views

question about empirical Bayes(EB)

In David Robinson's blog:Understanding empirical Bayes estimation (using baseball statistics) he used the hit ratio to fit the beta distribution as a prior distribution,Where hit rate = hits/total. ...
user avatar
  • 13
2 votes
1 answer
8k views

How to pick starting parameters for MASS:fitdist() with the beta distribution?

I have data set of ~700k yes/no events that I want to first aggregate on various features (e.g. group by average), always resulting in a 34 length vector. From there, I want to fit a beta ...
user avatar
  • 21
18 votes
0 answers
366 views

Empirical Bayes (In)Admissibility

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators. Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
user avatar
6 votes
2 answers
314 views

Can "cross-validation" be used to choose a prior?

To be clear, I doubt I am using the term "cross-validation" correctly here; what I am suggesting also seems similar to "boot-strapping" and "hyperparameter tuning". ...
user avatar
  • 5,689
2 votes
1 answer
115 views

Why the local Bayes fdr is greater than the Bayes FDR?

My question is related to empirical Bayes and large-scale inference. It is explained that the local Bayes false discovery rate (fdr) is greater than the Bayes false discovery rate (FDR). It is argued ...
user avatar
1 vote
0 answers
654 views

How to calculate the inverse cdf for a sample with unknown distribution?

I'm stuck with an exercise that I'll just write down first: Based on a sample $x_1,...,x_{47}$, that can be considered coming from an unknown distribution, we study a qq-plot where the empirical ...
user avatar
3 votes
1 answer
556 views

Empirical Bayes: method of moments

The model for the data is $X_{i}$ ~ $Bin(n_{i},\theta_{i})$ (iid, $i=1,...,k$). The prior distribution is $\theta_{i}$ ~ $Beta(\alpha,\beta)$. How do we choose (and deduct) moment estimators for $\...
user avatar
  • 97
5 votes
1 answer
1k views

Hyper-parameter estimation for Beta-Binomial Empirical Bayes

I am reading a paper Illustrating empirical Bayes methods and in the paper the author uses method of moments to derive the value of an estimate. In equation 17 the author gives the following marginal ...
user avatar
  • 943
2 votes
1 answer
74 views

Robbins estimate Empirical Bayes

From the compound sampling model where: $Y_i | \theta_i \sim Poi(\theta_i)$ The marginal distribution of $\theta_i$ is $G$, non-parametric. We get that the Bayes estimate of $\theta_i$ under ...
user avatar
  • 317
1 vote
1 answer
160 views

Empirical Bayes Estimation

Suppose $X_i$ conditioned on $\mu$ is iid $N(\mu, \sigma^2)$ and $\mu$ is distributed as $N(\mu_0, \tau^2)$. Is there a way to estimate $\tau^2$?
user avatar
  • 883
2 votes
1 answer
180 views

James-Stein Estimator with unequal numbers in groups

In the book Computer Age Statistical Inference the James-Stein estimator is introduced. Brad Efron runs through an example where batting averages are estimated from each players 90 at-bats. $$p_i\sim ...
user avatar
  • 943
27 votes
1 answer
2k views

How is empirical Bayes valid?

So I just finished reading a great book Introduction to Empirical Bayes. I thought that the book was great, but building priors from the data felt wrong. I was trained that you come up with an ...
user avatar
  • 943
7 votes
1 answer
310 views

Using empirical Bayesian estimation (Gamma-Poisson) to analyze high arrival counts (n ~= 5000)

Here's a problem I'm currently working on, as well as the empirical Bayesian approach I'm using. I'd like to make sure my approach is grounded in solid statistical theory. I have a set of entities $e=...
user avatar
  • 301
4 votes
1 answer
148 views

Expectation of inverse chi-square random variable (Efron, 2010)

Let $z_i|\mu\sim N(\mu,1)$ and $\mu\sim N(B,A)$ for $i=1,\dots,N$, the implication is that $z_i|B\sim N(B,A+1)$. Define $S=\|\textbf{z}\|^2$, and let $B=0$, then $S\sim(A+1)\chi^2_N$ since $\|\...
user avatar
0 votes
0 answers
190 views

Why does Empirical Bayes work in my simple case?

I have a problem where I am trying to classify data into two groups using a single parameter. The distribution of this parameter is Gaussian for two groups, so what I'm dealing with is two overlapping ...
user avatar
  • 41
6 votes
1 answer
268 views

Bootstrapping the data to set up a prior

I am using a Gaussian model with a conjugate Normal-Inverse-Wishart (NIW) prior, as described here. The advantage of this approach is that the marginal likelihood $p(y)$, which is what I am interested ...
user avatar
4 votes
1 answer
450 views

Empirical Bayes vs "non-informative" priors

I am familiar with the mechanics with both methods, but don't know what factors I should consider when choosing between these two approaches for adjusting a prior. I would imagine that, on a case by ...
user avatar
1 vote
1 answer
2k views

What is the empirical Bayes estimator for a gamma-Poisson with more than 1 observation for each Poisson parameter?

I am looking at the Wikipedia entry for empirical Bayes, but it's a bit confusing - it seems to me the solution must apply only to the case in which there's only $n=1$ sample $y$ for each $\theta$ and ...
user avatar
3 votes
1 answer
353 views

Empirical Bayes/MCMC references

I'm interested in references for running empirical Bayes (EB) in conjunction with MCMC. The closest thing I've found to what I'm looking at is a surprisingly recent and somewhat obscure paper ...
user avatar
  • 7,942
14 votes
1 answer
3k views

hierarchical Bayesian models vs. empirical Bayes

Would you consider the HBM vs EB to be two alternatives in which the hyperparameters are "in the game" of being sampled/estimated/etc.? There is clearly a connection between these two. Would you ...
user avatar
  • 662
14 votes
1 answer
3k views

Is there a connection between empirical Bayes and random effects?

I recently happened to read about empirical Bayes (Casella, 1985, An introduction to empirical Bayes data analysis) and it looked a lot like random effects model; in that both have estimates shrunken ...
user avatar
  • 141
24 votes
2 answers
4k views

Cross-validation vs empirical Bayes for estimating hyperparameters

Given a hierarchical model $p(x|\phi,\theta)$, I want a two stage process to fit the model. First, fix a handful of hyperparameters $\theta$, and then do Bayesian inference on the rest of the ...
user avatar
  • 1,570