1
$\begingroup$

I want to perform some sort of model evaluation of a multivariate distribution with the property that it is difficult/impossible to calculate the likelihood (of the whole model, you can do it for parts, but then they don't sum together well). Without being able to calculate the log-likelihood of the observations, I wouldn't be able to calculate the AIC, for instance. However, it is easy to generate random variables from the distribution.

Hence, I am wondering if there is any way to calculate anything like a log-likelihood for the observations using the simulated distribution?

I was thinking about splitting the multivariate distribution into marginals and a copula by following this (interpolate to get the CDF of the marginals and you're left with the copula values for each observation). For the marginals, I can make some assumptions to approximate the PDF and thus get values I can input into the likelihood. However, I'm not sure the same can be said for the copula as I'm not sure the easiest way to get to the multivariate pdf from the simulations. I had looked into multivariate kernel smoothing, but many applications do not allow for the large dimensions that I would require.

EDIT: I have tried to keep the everything general, but I'll provide a few more details on my specific issue. Suppose I fit some data with a multivariate normal (model 1). I then fit it with a factor model. The factor model, I can fit itself with a multivariate normal (model 2) or a more sophisticated distribution that is not easy to write-down (model 3). I want to evaluate which of these (or possible extensions with more or different factors) is best.

I assume that the factors explain the data reasonably well. This means that for either model 2 or 3 if I just calculate the log-likelihood as is typical, then it would look significantly better than model 1. However, this evaluation does not take into account that the factors themselves have distributions. In fact, you might be fooling yourself if you stop here. There might be correlation between the variables that the multivariate normal is picking up better than the factor model. You would have two alternatives (as far as I can see): 1) you could write the likelihood for the purposes of the AIC as p(observation|factor model)p(factor) or 2) you could convert it to a multivariate normal. Both 1 and 2 are doable. I imagine either of these would give me the same likelihood, but I would have to double-check. If they don't, then the second one is closer to what I would think of as comparable to model 1.

However, it's more difficult to do this for model 3 since we're under the assumption that it may not be easy to write the likelihood of the factor model (and you probably also couldn't convert it to one joint distribution either since only the sums of normal distributed variables are easy). By contrast, I could probably simulate such that I can produce the joint distribution that way. I'm wondering why I can't use this representation of the model to get something like a log-likelihood.

$\endgroup$
3
  • 2
    $\begingroup$ But you want to calculate log-likelihood for what? LL and AIC are used for model comparison and you don't seem to describe any model in here but rather a distribution... $\endgroup$
    – Tim
    Dec 19, 2014 at 21:26
  • $\begingroup$ @Tim, I've provided some more details in the question. $\endgroup$
    – John
    Dec 20, 2014 at 3:53
  • 1
    $\begingroup$ If you cannot evaluate the likelihood, indirect inference or approximate Bayesian computation (ABC) are a possibility. $\endgroup$
    – Xi'an
    Dec 20, 2014 at 8:25

1 Answer 1

1
$\begingroup$

Take a look at these implementations of Approximate Bayesian Computing (ABC) for when the likelihood can not be obtained:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.