# Why is the marginal liklihood term(the denominator) in Bayes' updation not a function of theta(parameters)?

The snippet from the book is pasted above. I don't understand how the denominator is not a function of theta as it is integrated over theta. Thank you.

-
Because it's a definite integral. – Glen_b Jun 16 '14 at 8:44

When you calculate a definite integral over some variable, and the limits don't contain that variable, the variable doesn't appear in the value. Indeed, you can substitute a different variable (integrate w.r.t. a different dummy) while leaving the integral unchanged.

So

$$\int_{-\infty}^{\infty} g(\theta) d\theta$$

isn't a function of $\theta$.

Edit - since more explanation is required (I am trying to convey an intuitive understanding, so this is a bit handwavy):

Note that when you do an indefinite integral, $\int g(x) dx$, the variable can appear in the result: $G(x)+c$. But when you do a definite integral $\int_a^b g(x) dx$, you substitute the limits into the antiderivative $(G(x)+c)|_a^b$, so every place the variable appeared in the antiderivative is replaced by the limits: $G(b)-G(a)$, leaving the variable absent from the result.

[This is not restricted to linear functions!]

-
I'm rusty on my integrals. Is this only true if the function is linear in $\theta$? – jabberwocky Jun 16 '14 at 9:07
sometimes its good to just provide a simple concrete example: What is $\int_0^5 x^2 dx$? What is $\int_0^\pi \sin{x}dx$? Are they functions of $x$? – bdeonovic Jun 16 '14 at 11:30

One source of confusion here may be that $\theta$ is used to denote two different things in the equation. $\theta$ appearing in the numerator is the same $\theta$ as in the left-hand-side, namely, the particular parameter values whose posterior density is obtained by substituting that particular $\theta$ on the right-hand-side.

Meanwhile, the $\theta$ in the denominator is just the variable of integration. As pointed out by @Glen_b, a definite integral is not a function of the variable of integration. Especially, $$\int f(\alpha) d\alpha = \int f(\theta) d\theta ,$$ as long as the limits (the area over which the integral is computed) are the same. It might be beneficial to write Bayes' rule using a different symbol in the denominator, but I guess the same symbol is used to denote that both refer to the same quantity, i.e., the same parameters of the model.

### An example

Consider tossing a (possibly biased) coin once, and observing heads. Let our parameter $\theta$ be the probability that the coin comes up heads on a single toss. Define prior (before the toss) as uniform $$p(\theta) = 1,~\theta \in [0,1].$$ Now, by definition of the parameter (and our data: heads once), $$p(D\mid \theta) = p(\mathrm{heads} \mid \theta) = \theta$$ Thus, the numerator is $p(\theta)p(D \mid \theta) = \theta$ when $\theta \in [0,1]$. The denominator is $$\int_0^1 p(\theta) p(\mathrm{heads} \mid \theta) d\theta = \int_0^1 \theta d\theta = \left. \frac{1}{2}\theta^2 \right|_{\theta=0}^{\theta = 1} = \frac{1}{2}.$$ As expected, this is not a function of $\theta$. It would be a function of the data $D$, but here for simplicity I have considered only a fixed $D$. Note that we could have denoted the parameter by any other symbol while computing the denominator: $$\int_0^1 p(\alpha) p(\mathrm{heads} \mid \alpha) d\alpha = \int_0^1 \alpha d\alpha = \left. \frac{1}{2}\alpha^2 \right|_{\alpha=0}^{\alpha = 1} = \frac{1}{2}.$$ The posterior is then $$p(\theta \mid D=\mathrm{heads}) = \frac{p(\theta)p(\mathrm{heads} \mid \theta)}{\int_0^1 p(\theta) p(\mathrm{heads} \mid \theta) d\theta} = \frac{\theta}{\frac{1}{2}} = 2\theta,$$ which is indeed proportional to $p(\theta)p(\mathrm{heads} \mid \theta)=\theta$.

-

The denominator in 11.1 is a function of $\theta$. However the denominator takes the same value irrespective of of which particular value of $\theta$ we are interested in because we intergrate over the whole range of possible values for $\theta$. The equation below 11.1 states that $P(\theta|D)$ is proportional to $P(\theta)L(\theta;D)$, not equal to it. i.e. $$P(\theta|D) = kP(\theta)P(D|\theta)$$ Here, the constant of proportionality, $k$ is the denominator form 11.1: $$k= \frac{1}{\int{P(\theta)P(D|\theta)d\theta}}$$ We can use this trick because often we are interested either in finding the $\theta$ that maximizes $P(\theta|D)$ rather than the value of $P(\theta|D)$ and so the constant doesn't matter, or we are interested in the ratio of $P(\theta|D)$ for two different values of theta, and so the $k$s cancel out.

This is a useful trick because the denominator can be difficult or impossible to calculate in many cases.

-