1
$\begingroup$

I've been thinking a lot about the relationships between various concepts like hypothesis testing, posterior distributions, and estimators.

If I understand correctly, a frequentist estimator $\hat\theta$ aims to approximate an unknown but constant $\theta$ based on some observed data $X$, or in other words it attempts to optimize some pointwise metric by marginalizing out $X$ (for example unbiasedness, $E_X[\hat\theta(X)-\theta|\theta]=0$ for all $\theta$). Confidence intervals are also implicitly conditioned on the unobserved $\theta$ (i.e. $P(\theta\in[a(X),b(X)]|\theta)=0.95$).

On the other hand, Bayesian posteriors attempt to marginalize out the unknown $\theta$ (as opposed to $X$) according to a prior distribution $P(\theta)$, replacing the "pointwise" estimate $\hat\theta$ with a single posterior distribution $P(\theta|X)$. The answer is simpler because we are assuming more than in the frequentist approach.

Both methods assume the same known data generating distribution $X\sim D(\theta)$.

Am I correct in my interpretation that frequentist methods try to marginalize out $X$ while Bayesian methods try to marginalize out $\theta$?

I found another (the same?) example of this "symmetry": https://en.wikipedia.org/wiki/Admissible_decision_rule#Generalized_Bayes_rules

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ How confidence intervals are conditioned on $\theta$? Moreover, in frequentist setting $\theta$ is not a random variable, so you couldn't condition on it. It also does not have distribution. $\endgroup$
    – Tim ♦
    May 30, 2019 at 19:42
  • $\begingroup$ If you think of theta as a random variable and condition everything on it, it becomes a constant. I was just trying to emphasize that $\theta$ is fixed and you make a separate claim about $\hat\theta$ for each possible value of $\theta$ in the frequentist setting. $\endgroup$
    – Akababa
    May 30, 2019 at 19:45
  • $\begingroup$ What is claimed "for all values of $\theta$"? What all values? If population is single mean, then you condition on constant? $\endgroup$
    – Tim ♦
    May 30, 2019 at 19:48
  • $\begingroup$ It's a claim about the estimator as a function from X to $\mathbb{R}$, when we say it's unbiased we mean $E_X[\hat\theta]=\theta$ for all values of $\theta$ (or it may only be unbiased for some values of $\theta$). $\endgroup$
    – Akababa
    May 30, 2019 at 19:53
  • 1
    $\begingroup$ Because the frequentist estimator does not conceive of $\theta$ as having any kind of probability distribution, it makes no sense to talk of marginal distributions. When the frequentist can justify adopting a probability model for $\theta,$ her techniques include everything that the Bayesian theory uses. $\endgroup$
    – whuber ♦
    May 30, 2019 at 20:03

1 Answer 1

Reset to default
0
$\begingroup$

In Bayesian setting, you are considering the prior distribution of $\theta$, this does let you use Bayes theorem and calculate the conditional probabilities. In frequentist setting you end up with estimate that depends on the observed data. In frequentist setting you are not "marginalizing out" the data unless you observed the whole population, otherwise this is still an estimate given single sample. So I would argue that this intuition is not correct.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.