6
$\begingroup$

I see no equations for the following, so I'm not sure exactly what they are talking about:

"For each model, we determine the best fit parameters from the peak of the N-dimensional likelihood surface. For each parameter in the model we also compute its one dimensional likelihood function by marginalizing over all other parameters."

How do you obtain a one-dimensional likelihood function by marginalizing over all other parameters?

Improve this question
$\endgroup$
2
  • $\begingroup$ Is my answer helpful or are you looking for other ways of marginalization? Are you expecting an answer on how to do the integration or summation technically? $\endgroup$ – gwr Nov 26 '15 at 18:32
  • $\begingroup$ @gwr This was helpful. I am interested in how one technically does the integrals though, of course. $\endgroup$ – ShanZhengYang Nov 26 '15 at 18:50
1
$\begingroup$

Say you have the state of Information $I$, some observations $\{y_i\}$ and some parameters $\theta_p$ where $p \in \{1,2, \ldots n\}$ then in the continuous case you get the marginal likelihoods from the joint likelihood $p(\theta_1,\theta_2, \ldots \theta_n |\{y_i\},I)$ by integration:

\begin{align*} p(\theta_k|\{y_i\},I) &=\int_{-\infty}^{+\infty} \ldots \int_{-\infty}^{+\infty} p(\theta_1,\theta_2, \ldots \theta_n |\{y_i\},I)\,d\theta_{p_1} \cdots \,d\theta_{p_{n-1}} \end{align*}

where $p_j \in \{1,2, \ldots n\}\setminus k$

In the discrete case you will have sums instead of integrals as you can see here.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ That gives one way of marginalizing, there are others. $\endgroup$ – kjetil b halvorsen♦ Nov 26 '15 at 12:59
  • 2
    $\begingroup$ When integrating directly the likelihood with respect to the Lebesgue measure over the other parameters, you make an arbitrary choice of dominating measure, which means that your definition of a profile likelihood is not invariant by reparameterisation. $\endgroup$ – Xi'an Nov 26 '15 at 19:58
  • $\begingroup$ @Xi'an Thank you. That is what I call a constructive critique and I have learned something. What I am doing here is what I most often see done for a joint posterior distribution; so is the issue of invariance mainly a concern for priors or likelihoods? $\endgroup$ – gwr Nov 26 '15 at 20:38
  • 1
    $\begingroup$ The lack of invariance is an issue for the marginal likelihood: if you substitute for $\theta_{-k}$ a bijective transform of $\theta_{-k}$ that does not modify $\theta_k$ the resulting marginal as defined above will not be the same function of $\theta_k$. In other words, the Jacobian gets in the way and points out the influence of the dominating measure. $\endgroup$ – Xi'an Nov 26 '15 at 20:40
2
$\begingroup$

You should always reference your quotes.

The only reference on the Internet I found with this quote

"For each model, we determine the best fit parameters from the peak of the N-dimensional likelihood surface. For each parameter in the model we also compute its one dimensional likelihood function by marginalizing over all other parameters." (p.3)

is an cosmology arxiv paper on WMAP by Spergel et al. (2003). If you turn the page after this quote, you will find an equation defining the expectation under the "marginal likelihood" as $$<α_i> = \int d^N α\mathcal{L}(α)α_i\,.$$ This means that the "marginal likelihood" is $$\int d^{N-1} α_{-i}\mathcal{L}(α)\,.$$

If I quote from the previous sentences of the paper as well

"For each model studied in the paper, we use a Monte Carlo Markov Chain to explore the likelihood surface. We assume flat priors in our basic parameters, impose positivity constraints on the matter and baryon density (these limits lie at such low likelihood that they are unimportant for the models. We assume a flat prior in τ , the optical depth, but bound τ < 0.3. This prior has little effect on the fits but keeps the Markov Chain out of unphysical regions of parameter space. (p.3)"

this means that the authors take a Bayesian stance with a flat prior in their parametrisation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.