Questions tagged [empirical-bayes]
Procedures for statistical inference in which the prior distribution is estimated from the data.
60
questions
0
votes
0
answers
13
views
1
vote
0
answers
28
views
How to calculate the log marginal distribution in Tweedie's formula?
2018 Bayes, oracle Bayes, and empirical Bayes
2021 Empirical Bayes: Concepts and methods
Two modeling strategies for empirical Bayes estimation
2013 Empirical Bayes modeling, computation, and ...
- 33
5
votes
1
answer
116
views
Doing empirical Bayes with improper prior - marginals that do not exist?
I am considering a Bayesian linear model for which the prior is not proper.
The model is as usual $y = X \theta + w$ where $w \sim N(0, \sigma^2)$, and $\theta, \sigma^2$ are unknown.
The distribution ...
2
votes
0
answers
28
views
Is there a proper way to combine two beta prior distributions?
I'm working on a research project which involves the assessment of NHL goalie performance (save percentage) using empirical Bayes. I found that the goalie distribution for career games played was ...
- 121
2
votes
1
answer
203
views
Is Bayesian model selection with empirical parameter priors sound?
Overview
I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.
Example Scenario: Coins
Suppose I have a collection of ...
- 2,261
4
votes
1
answer
75
views
Possibility priors in Bayesian analysis?
A couple of trains of thought have come together for a model I am designing. Let's start with the first part: Bayesian inference doesn't update strongly enough.
One of the parameters $\theta$ is an ...
- 7,008
2
votes
1
answer
108
views
Trying to understand if a model is an empirical Bayes or not
I am trying to understand if the model published in section 4.1.1 here is or is not an Empirical Bayes model (which the author claims it is). Or, maybe, if it is a valid one or not. The model looks as ...
- 157
3
votes
1
answer
378
views
Is it wrong to use sufficent statistc estimated from the data as a prior in Bayesian data analysis?
First I want to state that I got unexpected feedback from a reviewer in regard to my question and I am simply interested in others' views in this regard (I have already sent in my rebuttal).
Suppose ...
- 157
1
vote
0
answers
44
views
Why are fisher-scoring estimates of the fixed-effects not used to calculate empirical bayes estimates of random-effects? Are they in-admissible?
Why is it not practiced using estimates of fixed-effects from fisher scoring used to calculate GEE coefficients to estimate random-effects via empirical bayes? We have another estimate of $\theta$ in ...
user318514
1
vote
0
answers
138
views
In Bayesian hierarchical models, what is the difference between an Empirical Bayesian approach to parametrising priors vs using flat hyperpriors
Say I have a simple hierarchical model, where:
$y_{g,i} = \beta_g x_{g,i} + e_{g,i}$
where $g$ represents the group, $i$ represents the individual within the group, and $e$ is the error. So the ...
- 71
0
votes
0
answers
100
views
How to combine priors for empirical bayes
I am trying to estimate $Z_{i} = P(Y=1 | A=a_{i}, B = b_{i}, C =c_{i})$ using something like empirical Bayes as in http://varianceexplained.org/r/empirical_bayes_baseball/
aggregating a massive amount ...
- 8,100
1
vote
0
answers
83
views
Which marginal likelihood is maximized in empirical Bayes?
I am reading up on empirical Bayes, and I am finding it really hard to understand what exactly is going on.
As far as I understand, it is based on maximizing the marginal likelihood of the data (so ...
- 11
4
votes
0
answers
59
views
Why Are Empirical Bayes Methods Not Considered "Controversial"? [duplicate]
I was reading about Empirical Bayesian Methods and came across the following:
My Question: As this text explains, I have often heard that the priors used in Bayesian Methods should be decided prior ...
1
vote
0
answers
75
views
Empirical bayes estimation of variograms for multiple variables
I am looking to estimate variograms, or spatial correlation matrices on all columns of a data matrix $\mathbf{X}_{n\times p}$ with $p>n$. The matrix of spatial coordinates $\mathbf{Y}_{n\times 2}$ ...
- 1,617
1
vote
0
answers
88
views
Empirical Bayes prediction
I've been working with a GLMM in SAS. Using the proc GLIMMIX procedure, I've extracted the EB estimates of the random-effects.
I was looking into empirical Bayes estimates and their predictive power.
...
- 21
0
votes
1
answer
47
views
How to manually adjust predicted probabilities from lm based on prior lognormal distribution parameters?
@drob showed a great example of adjusting batting averages using a beta-based prior distribution. He used a prior calculated Beta distribution to adjust batting averages individually, and
it’s as ...
- 227
3
votes
1
answer
148
views
Is empirical Bayes estimation still possible without common parameters?
In An Introduction to Empirical Bayes Data Analysis by George Casella (1985), it is given that
\begin{align}
X|\theta &\sim N(\theta,\sigma^2) \\
\theta &\sim N(\mu,\tau^2) \\
\theta|X &\...
- 4,125
4
votes
1
answer
198
views
Is empirical Bayes an iterative scheme?
From the Wikipedia article on the empirical Bayes method (emphasis mine):
In, for example, a two-stage hierarchical Bayes model, observed data $y=\{y_{1},y_{2},\dots ,y_{n}\}$ are assumed to be ...
- 4,125
1
vote
0
answers
28
views
Nonparameteric Empirical Estimator For Stochastic Process
Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
- 544
1
vote
0
answers
45
views
Limiting distribution of iterative applications of Bayes' rule
The question
Suppose we iteratively use the posterior as the prior on the same data.* What is the limiting distribution of the posterior?
Let's make that precise. The data $X$ and the likelihood ...
- 580
0
votes
0
answers
839
views
How to estimate beta distribution parameters using a beta binomial with empirical bayes
I would like to estimate parameters for a beta distribution using a maximum likelihood approach in python (as mentioned here). I can do this for a beta:
...
2
votes
0
answers
65
views
Empirical bayes approach to hypothesis testing
Empirical Bayes seems to be quite successful in point estimation. A classical example would be the Robbins' formula to estimate the Poisson rate. Robbins also published a paper about the application ...
- 88
0
votes
0
answers
204
views
Generating Poisson-Gamma data for empirical parameter estimation in r
I'm currently trying to learn parameter estimation and want to illustrate and compare different credibility methods (limited fluctuation, Bühlmann, Bühlmann-Straub) using both Bayesian and Empirical ...
- 41
4
votes
1
answer
601
views
Using the MLE to select the prior distribution...empirical Bayes?
It was requested that I read the following article for work:
https://support.sas.com/resources/papers/proceedings15/1400-2015.pdf
In Case II, the author starts by doing two things:
First, he computes ...
- 966
1
vote
0
answers
50
views
Empirical Bayes Estimation: Different transformation, fully different results
I would be grateful for some advice regarding the following problems:
As I reviewed some empirical papers, I learned that most of them use a relatively small number of observations for performing ...
- 11
2
votes
1
answer
1k
views
Beta-Binomial regression or Poisson-Gamma model to account for uncertainty in (empricial Bayesian) prior? Explained in simple terms?
I have a dataset of $m$ individuals. For each individual $m$ I have $n_m$ (binomial ) observations with $s_m$ corresponding to the number of 'successes'.
I use this data to fit a beta-binomial ...
5
votes
1
answer
294
views
Admissible Empirical Bayes Examples
I would like to hear about a few simple empirical bayes estimators that are admissible for high (i.e. at least 3) dimensional parameter space.
What are some textbook lollipop examples to study for ...
- 4,106
4
votes
2
answers
166
views
Accounting for uncertain information (few observations) in a prior (empirial Bayes)
I did not really know how to choose an adequate title for this question, so please feel free to change it.
I have a weird case wherein frequentist and Bayesian philosophies come together. I am ...
5
votes
1
answer
459
views
Likelihood of linear mixed effects model
Consider the following model
$$\left \{
\begin{array}{l}
y_i = x_i\beta + z_ib + \varepsilon_i,\\\\
b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2),
\end{array}
\right....
- 823
2
votes
0
answers
106
views
Replicating a Tweedie corrected experiment from Computer Age Statistical Inference
I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411.
In this experiment 100 datasets are populated normal random ...
- 1,921
5
votes
2
answers
389
views
Is appropriate to use empirical Bayes (EB) in this way?
Background. I have data from a study where participants make a series of judgments (a series of decisions with a binomial outcome, either $y=1$ or $y=0$). I have a model of the underlying decision-...
- 3,143
1
vote
1
answer
417
views
Poisson-gamma posterior mean expectation
Let's have a gamma prior $\lambda\sim \operatorname{Gamma}(a,b)$ (mean: $\frac{a}{b}$)
With Poisson data $Y\mid \lambda\sim \operatorname{Pois}(N\lambda)$ (mean: $N\lambda$)
The posterior is $\...
- 25
1
vote
1
answer
140
views
Empirial Bayes for means
I have data for US ~3100 counties where the variable is a mean score based on a sample. However, for many small counties, the sample size is quite small (like 5), and so these mean values fluctuate a ...
- 439
1
vote
0
answers
155
views
Is expectation maximization an example of empirical Bayes?
I don't think I truly understand what methods are classified as "empirical Bayes". Is expectation maximization considered an example of this?
- 71
2
votes
1
answer
176
views
How do Bayesian hierarchical models adaptively learn the prior?
It seems the main difference between a hierarchical and a non hierarchical model is that the hierarchical model learns the prior. That is it adaptively comes up with a regularizing prior to be applied ...
- 261
3
votes
1
answer
2k
views
Learning prior distribution from data
Suppose I have a dataset. How can I learn the prior distributions of the parameters of a model from this data? I want to learn the prior from this data in order to use them in a Bayesian model. Sorry ...
- 41
0
votes
0
answers
154
views
question about empirical Bayes（EB）
In David Robinson's blog：Understanding empirical Bayes estimation (using baseball statistics) he used the hit ratio to fit the beta distribution as a prior distribution，Where hit rate = hits/total. ...
- 33
5
votes
1
answer
11k
views
How to pick starting parameters for MASS:fitdist() with the beta distribution?
I have data set of ~700k yes/no events that I want to first aggregate on various features (e.g. group by average), always resulting in a 34 length vector. From there, I want to fit a beta ...
- 51
19
votes
0
answers
465
views
Empirical Bayes (In)Admissibility
Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators.
Nevertheless, there is a good reason to use Empirical Bayes in many cases, and ...
- 4,106
7
votes
3
answers
473
views
Can "cross-validation" be used to choose a prior?
To be clear, I doubt I am using the term "cross-validation" correctly here; what I am suggesting also seems similar to "boot-strapping" and "hyperparameter tuning". ...
- 6,119
2
votes
1
answer
166
views
Why the local Bayes fdr is greater than the Bayes FDR?
My question is related to empirical Bayes and large-scale inference. It is explained that the local Bayes false discovery rate (fdr) is greater than the Bayes false discovery rate (FDR).
It is argued ...
- 109
1
vote
0
answers
713
views
How to calculate the inverse cdf for a sample with unknown distribution?
I'm stuck with an exercise that I'll just write down first:
Based on a sample $x_1,...,x_{47}$, that can be considered coming from an unknown distribution, we study a qq-plot where the empirical ...
- 139
3
votes
1
answer
770
views
Empirical Bayes: method of moments
The model for the data is $X_{i}$ ~ $Bin(n_{i},\theta_{i})$ (iid, $i=1,...,k$).
The prior distribution is $\theta_{i}$ ~ $Beta(\alpha,\beta)$.
How do we choose (and deduct) moment estimators for $\...
- 97
8
votes
1
answer
2k
views
Hyper-parameter estimation for Beta-Binomial Empirical Bayes
I am reading a paper Illustrating empirical Bayes methods and in the paper the author uses method of moments to derive the value of an estimate. In equation 17 the author gives the following marginal ...
- 1,921
2
votes
1
answer
97
views
Robbins estimate Empirical Bayes
From the compound sampling model where:
$Y_i | \theta_i \sim Poi(\theta_i)$
The marginal distribution of $\theta_i$ is $G$, non-parametric.
We get that the Bayes estimate of $\theta_i$ under ...
- 317
3
votes
2
answers
248
views
Empirical Bayes Estimation
Suppose $X_i$ conditioned on $\mu$ is iid $N(\mu, \sigma^2)$ and $\mu$ is distributed as $N(\mu_0, \tau^2)$. Is there a way to estimate $\tau^2$?
- 993
3
votes
1
answer
230
views
James-Stein Estimator with unequal numbers in groups
In the book Computer Age Statistical Inference the James-Stein estimator is introduced. Brad Efron runs through an example where batting averages are estimated from each players 90 at-bats.
$$p_i\sim ...
- 1,921
29
votes
1
answer
2k
views
How is empirical Bayes valid?
So I just finished reading a great book Introduction to Empirical Bayes.
I thought that the book was great, but building priors from the data felt wrong. I was trained that you come up with an ...
- 1,921
1
vote
0
answers
76
views
How to apply empirical Bayes to multi-class classification?
We can use an empirical Bayes framework to iteratively refine the least-squares fitting of some model $f$ parameterized by weights $\textbf{w}$. This can be done by presuming a Gaussian prior on the ...
- 785
7
votes
1
answer
421
views
Using empirical Bayesian estimation (Gamma-Poisson) to analyze high arrival counts (n ~= 5000)
Here's a problem I'm currently working on, as well as the empirical Bayesian approach I'm using. I'd like to make sure my approach is grounded in solid statistical theory.
I have a set of entities $e=...
- 301