2
$\begingroup$

Although I am completely new to Bayesian Analysis I struggle sometimes when trying to investigate some intersections between Bayesian and Frequentist analysis. I would like to discuss the different implications based on the following simple example:

Let $y=(y_1,\ldots,y_N)$ were $y_i|\mu \sim N(\mu,1)$, $\mu\in\mathbb{R}$. Hereby $\mu$ is not known to us. A Frequentist would compute $\hat{\mu}=\frac{1}{N}\sum_{i=1}^{N}y_i$. For the estimator $\hat{\mu}$ it holds that $\hat{\mu} \sim N(\mu,\frac{1}{N})$.

In a Bayesian approach we could assign a normal prior for $\mu$ with infinite variance $\mu\sim N(\mu_0,\lambda^{-1}), \lambda\rightarrow \infty$ and obtain a normal distributed posterior density $\mu|y\sim N(\hat{\mu},\frac{1}{N})$.

I am wondering how $\hat{\mu} \sim N(\mu,\frac{1}{N})$ and $\mu|y\sim N(\hat{\mu},\frac{1}{N})$ can be interpreted and what different implications you can draw out of those formulas. Is it a correct Bayesian interpretation to state that after observing our data $y$ the variance term $\frac{1}{N}$ reflects the 'uncertainty' that is incorporated in choosing $\hat{\mu}$ as our decision rule? On the other hand side what does the Frequentist approach tells us regarding the 'true' parameter? Each and every comment is welcome!

Improve this question
$\endgroup$
2
  • $\begingroup$ special case of stats.stackexchange.com/questions/31867/… ? $\endgroup$
    – Christoph Hanck
    Jun 19, 2015 at 10:40
  • 1
    $\begingroup$ Surely this question is related to the general discussion about interpreting Bayesian vs. Frequentist probabilites. However, I do struggle in transferring the philosophical considerations into an example like the one given above. I would love to see what the Bayesian interpretation of uncertainty and the Frequentist idea of asymptotic behaviour implies for these two representations of $\hat{\mu}$ and $\mu|y$. $\endgroup$
    – Stefan Voigt
    Jun 19, 2015 at 11:05

1 Answer 1

Reset to default
4
$\begingroup$

So, I welcome any comments or corrections, it's been a while since I've sat in front of a textbook.

Insofar as I've always thought of it, the frequentist isn't as interested in the distribution of $\hat{\mu}$ as the Bayesian is of $\mu \mid y$ (I'm totally going to get flamed for that statement). Why do I say this?

  • In the frequentist view of things, $\mu$ is a degenerate random variable, and $\hat{\mu}$ is our best guess at $\mu$. Hence, we may use the (implicit) distribution of the estimator as a means of handling the error of our guess, but we can't really interpret $$P( \hat{\mu} \in [a,b]) = p(a,b)$$ as something like

    The probability that $\mu$ lies in $[a,b]$ is $p(a,b)$.

    Instead, $p(a,b)$ encodes our uncertainty in $\hat{\mu}$ and -- this is my opinion -- this doesn't really help us get a handle on $\mu$...

  • In the bayesian view, however (as you've identified), $\mu$ is a (nondegenerate) random variable, and our collection of data, $y$, affords us the ability to say things like $$P(\mu \in [a,b] \mid y) = p(a,b)$$

    The probability that $\mu$ lies in $[a,b]$ is $p(a,b)$.

... I'm not sure if this helps, but leave a comment and I'll cleanup/elaborate as necessary!

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your great answer.I think as $\mu$ is unknown the amount of information we obtain regarding $\mu$ should be the same for both of the Probability Interpretations and therefore $P(\hat{\mu} \in [a,b])=p(a,b)$ does not really help to learn something about $\mu$ that is not incorporated in $\mu|y$. For me the representation $\hat{\mu}\sim N(\mu,\frac{1}{N})$ just states our observed variable $\hat{\mu}$ is one realization of a Normal variable with unknown mean. Therefore we know nothing about $\mu$. But what is then the value of the distribution of $\hat{\mu}$ for the Frequentist? $\endgroup$
    – Stefan Voigt
    Jun 19, 2015 at 11:25
  • 1
    $\begingroup$ So, let me start off backwards: for me that $\hat{\mu}$'s distribution is specified relative to an unknown quantity makes it pretty unhelpful if I'm trying to do inference on $\mu$. Maybe other people have less obvious things they want to infer... Me, on the other hand, I'm a simple guy. If I'm trying to estimate $\mu$, I'd like some easily interpret-able statements about it; sadly, frequentist confidence intervals are hard for non-statisticians to interpret... $\endgroup$
    – StevieP
    Jun 22, 2015 at 21:34
  • 1
    $\begingroup$ Towards your 3rd sentence, however: I'm not sure how $P(\hat{\mu} \in [a,b])$ is incorporated in $\mu \mid y$... Sure, $\mu \mid y \sim N(\hat{\mu}, N^{-1})$, but I'm not seeing an obvious connection aside from the fact that both mathematical statements reference $\hat{\mu}$. $\endgroup$
    – StevieP
    Jun 22, 2015 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.