0
$\begingroup$

I know $P(B|A) = \frac{P(A|B)P(B)}{P(A)}$, where $P(B)$ is known as a prior probability, $P(B|A)$ is a posterior probability and $P(A|B)$ is called the likelihood. We can interpreting $B$ as the model parameters and $A$ as the data, to estimate the parameters of a model from a sample. What I am confused is why we can regard $P(B)$ as a constant and $P(A)$, although it is unknown, also as a constant?

I am also stuck at a specific example: let the prior probability of $\sigma$ be $p(\sigma)$, and the $x_i$ are drawn from an $N(0, \sigma^2)$ population. I know $p(\textbf{x}|\sigma)$ is proportional to $\sigma^{-n}e^{-ns^{2}/\sigma^2}$, where s is the standard deviation of the sample, but why $p(\textbf{x})$ is proportional to $\frac{1}{s^n}$?

$\endgroup$
  • 3
    $\begingroup$ Could you explain in what sense a probability would not be a constant? After all, it is defined to be a number and--absent any specification of what that number might depend on--we shouldn't expect numbers to vary! BTW, please check your post for typographical errors: the formula at the outset is clearly wrong, leading one to wonder about the mathematical expressions later in the question. $\endgroup$ – whuber Jul 8 at 18:16
  • 1
    $\begingroup$ Also you do not specify the prior in the Normal example. Deriving $p(\mathbf x)$ is impossible. $\endgroup$ – Xi'an Jul 8 at 18:34
  • $\begingroup$ @Xi'an Can you help me a little bit more about that? My textbook (R. Lupton) gives directly $p(\textbf{x})$ is inversely proportional to $s^{n}$, which I do not understand, and $p(\sigma)$ can later be derived as $1/\sigma$. $\endgroup$ – consideration Jul 8 at 21:28
0
$\begingroup$

What I am confused is why we can regard P(B) as a constant and P(A), although it is unknown, also as a constant?

Notice the posterior P(B|A) is also a distribution, so it has to integrate to 1. Given this, the P(A) aka P(Data) must be a constant to make the $\frac{P(A|B)P(B)}{P(A)}$ integrate to 1. P(Data) is usually hard to calculate, and luckily we usually don't have to calculate it because it has nothing to do with the parameter we are evaluating and is a constant.

where s is the standard deviation of the sample, but why p(x) is proportional to $\frac{1}{s^n}$

notice p(x) has no $\sigma$ in it, and thus it has to be a constant (proportional to $\frac{1}{s^n}$) to make the posterior distribution $p(\sigma|x)$ integrate to 1

$\endgroup$
  • $\begingroup$ Because $P(\textbf{x}|\sigma)$ is proportional to $\sigma^{-n}*e^{-ns^2/\sigma^2}$, and $P(\sigma) = 1/\sigma$, $P(\textbf{x})$ is proportional to $\int_{0}^{\infty} \sigma^{-n-1} e^{-ns^2/\sigma^2} d\sigma$. I guess the author means the this result is of the form that looks like $s^n$ instead of exactly $s^n$ multiplying some numerical constants, because when I refer to some useful (Gaussian) definite integrals, I do not find a formula that can evaluate this integral exactly. $\endgroup$ – consideration Jul 9 at 19:04
0
$\begingroup$

In bayesian estimation the prior and posterior distribution works together. You can check on wikipedia the conjugate prior https://en.wikipedia.org/wiki/Conjugate_prior.

The ideas is to give a prior such that the posterior stay in the family of distribution you want to work with. You can see why he takes the prior proportional to what you say in the table of the wikipedia page.

After $P(A)$ and $P(B)$ are not constant, its just we are often interested by estimating a parameter. And we just need to know your distribution is proportional to the parameter you want to estimate. You write $p(\mathbf{x}|\sigma) $ is proportional to a function of $\sigma$, and in this setting $\sigma$ is your target. So you can compute MAP (maximum a posteriori) estimation just knowing it is proportional. If you are motivated you can compute all.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.