3
$\begingroup$

Suppose we have data $x_i, i=1,2,3,...n$ that are dependent and identically distributed with marginal $f(\cdot|\alpha)$. If we model this with the likelihood

$ L = c(F(x_1|\alpha),F(x_2|\alpha),...F(x_n|\alpha)|\theta)\prod_{i=1}^n f(x_i|\alpha) $

and the dependence parameter $\theta$ is known, can we apply some variant of the Expectation Maximization algorithm to estimate $\alpha$ using an iterative procedure with relatively simple steps?

For instance, I considered a simple problem with exponential marginals and Gaussian copula (with known correlation), and did something procedural (and hokey). I introduced the unknown independent samples $\tilde{x}_i, i=1,2,...,n$ which you would compute by knowing the correct value of $\alpha$, mapping the $x_i$ to correlated Gaussians $y_i = \Phi^{-1}(F(x_i|\alpha))$ and then "undoing" the correlations $z = B^{-1}y$ and mapping the $z$ forward again to produce the independent $\tilde{x_i}=F^{-1}(\Phi(z_i)|\alpha)$. Here $C=BB^T$ is the correlation matrix. Here $\Phi$ is the (0,1)-normal cdf. If you turn this into an iterative procedure, using $x$ as the initial guess for the independent data, it seems to produce a series that (at least in my trials) converged. However, the whole thing is doubtful since it depends entirely on what you choose for $B$ (only defined up to a unitary matrix). I think it's the unitary invariance of the Gaussian hitting you when you try to basically do an inversion.

Is there an obvious way to turn this kind of problem into a sane iterative procedure using simple steps like the EM? I feel like I'm missing a simple trick.

$\endgroup$
5
  • 2
    $\begingroup$ Cross-posted to mathoverflow mathoverflow.net/questions/105596 $\endgroup$ Commented Aug 27, 2012 at 3:14
  • $\begingroup$ Just read a comment somewhere about not cross-posting ... is this true? I should mostly use mathoverflow then right? I'm not quite sure how to decide where to post. I would rather use "tags" than "sites" to distinguish mathy content. $\endgroup$
    – safetyduck
    Commented Aug 27, 2012 at 3:35
  • 1
    $\begingroup$ I would ask this question of statisticians, myself. I don't know if we have experts in the EM algorithm at MO. A search of MO turns up only a handful of questions (17 non-unique results, including your question) either closed or with no (substantive) answers. A search of stats.sx turns up 782 results. I'll leave it to you to decide where to ask the questions. $\endgroup$ Commented Aug 27, 2012 at 5:16
  • $\begingroup$ Actually, the question of where to post is quite interesting. A crude picture might be that mass streams like mathoverflow get faster responses but specialized streams like stats.stackexchange will get more complete "quality" answers on a longer time horizon. I can see why one would want to keep cross-posting down but I can also see that if sites offer different things, the same question may in fact represent demand for entirely different responses. Anyway, I'll request to take one of these down soon. $\endgroup$
    – safetyduck
    Commented Aug 27, 2012 at 13:18
  • $\begingroup$ This question could be closed ... a very good answer was received on mathoverflow. I still find it strange that there are actually different sites for overlapping, fuzzy topics but maybe this works well for some people. Tagging and statistical assignment might work better with the special subsites existing more as particular "views" of the data. We think of mathematicians and scientists as specialists, but I think many solutions and innovations come precisely from reaching outside one's known scope. $\endgroup$
    – safetyduck
    Commented Nov 22, 2012 at 13:35

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.