I read that in Bayes rule, the denominator $\Pr(\textrm{data})$ of

$$\Pr(\text{parameters} \mid \text{data}) = \frac{\Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\text{parameters})}{\Pr(\text{data})}$$

is called a normalizing constant. What exactly is it? What is its purpose? Why does it look like $\Pr(data)$? Why doesn't it depend on the parameters?

  • 6
    $\begingroup$ When you integrate $f(\text{data}|\text{params})f(\text{params})$, you are integrating over the parameters and so the result has no term depending on the parameters, in the same way that $\int_{x=0}^{x=2}xy\;dx = 2y$ does not depend on $x$. $\endgroup$ – Henry Jun 20 '11 at 18:57

The denominator, $\Pr(\textrm{data})$, is obtained by integrating out the parameters from the join probability, $\Pr(\textrm{data}, \textrm{parameters})$. This is the marginal probability of the data and, of course, it does not depend on the parameters since these have been integrated out.

Now, since:

  • $\Pr(\textrm{data})$ does not depend on the parameters for which one wants to make inference;
  • $\Pr(\textrm{data})$ is generally difficult to calculate in a closed-form;

one often uses the following adaptation of Baye's formula:

$\Pr(\textrm{parameters} \mid \textrm{data}) \propto \Pr(\textrm{data} \mid \textrm{parameters}) \Pr(\textrm{parameters})$

Basically, $\Pr(\textrm{data})$ is nothing but a "normalising constant", i.e., a constant that makes the posterior density integrate to one.

  • 2
    $\begingroup$ @nbro: I mean Pr(data) = integral over the parameters of Pr(data, parameters) $\endgroup$ – ocram Mar 8 '18 at 6:24
  • $\begingroup$ What do you mean by 'P(data) is generally difficult to calculate in a closed-form'? $\endgroup$ – unicorn Aug 18 '20 at 7:56
  • $\begingroup$ @unicorn: To calculate P(data), one has to integrate P(data, parameters) over the parameters. This task is generally difficult. $\endgroup$ – ocram Aug 19 '20 at 5:31

When applying Bayes' rule, we usually wish to infer the "parameters" and the "data" is already given. Thus, $\Pr(\textrm{data})$ is a constant and we can assume that it is just a normalizing factor.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.