4
$\begingroup$

I have a certain computational biology problem I wish to model. Say I have a vector $\vec{f}$ that yields $\vec{p}$, of which the explicit form amounts to picking $\vec{p}$ as an eigenvector from an eigenvalue-problem with the largest associated eigenvalue.

$\vec{p}$ is a probability vector, i.e. its components sum to 1. I now have a multinomial distribution with $\vec{p}$ as the parameter. The data I have consists of $N$ samples from this multinomial distribution.

As is the case in statistics, I do not know $\vec{p}$ (or $\vec{f}$ for that matter). Say I can construct some confidence interval for $\vec{p}$ of the multinomial distribution (there are a number of methods in literature), how do I transform this confidence interval to $\vec{f}$? The function is defined as $\vec{f} = \vec{h}(\vec{p}) = \left( Q^T diag\left( \vec{p} \right) \right)^{-1} \vec{p}$, where $Q$ is a stochastic transition matrix, i.e. its rows sum to 1.

I know that if the transformation is monotonic, the boundaries maybe simply be transformed to yield a confidence interval for $\vec{f}$. So far I have linearised $\vec{h}(\vec{p})$ around $\hat{p}$ and used this with okayish results, but there remain some substantial departures from 'optimal' confidence intervals as observed from non-parametric bootstrap samples.

Ultimately, I would like to work in this framework, and if possible not go to Bayesian (or non-parametric bootstrapping), as the dimensionality of the problem will scale up (realistically $\vec{f}$ might be 1000-dimensional or so).

Thanks for any suggestions.

$\endgroup$
  • $\begingroup$ OK, thanks. Please tell us whether this is a correct interpretation: given a known stochastic matrix $Q$ and a confidence interval for $\vec{p}$ based on multinomial observations (with about $1000$ classes), you wish to compute a confidence interval for $\left( Q^T diag\left( \vec{p} \right) \right)^{-1} \vec{p}$? $\endgroup$ – whuber Jun 16 '12 at 12:11
  • 1
    $\begingroup$ That's it, spot on. $\endgroup$ – Jonas Müller Jun 16 '12 at 12:14
  • $\begingroup$ There is the Dirichlet conjugate prior for the Bayesian approach, hence the dimensionality is not a problem. $\endgroup$ – Stéphane Laurent Jun 16 '12 at 13:58
  • $\begingroup$ True, for the case in $\vec{p}$, but this needs to be transformed to $\vec{f}$ (which is what I'm really interested in). As I mentioned earlier, the inverse problem is known but it amounts to an eigenvalue problem, which would be needed if you wanted to transform the pdf in $\vec{p}$ to a pdf in $\vec{f}$, for which AFAIK there is no analytical solution. Furthermore, say we sample from the posterior of $\vec{p}$ in such a case, you would still have to transform the sample for $\vec{f}$, and doing that might be too computationally demanding... $\endgroup$ – Jonas Müller Jun 16 '12 at 14:13
  • $\begingroup$ How about the multivariate delta method ? (en.wikipedia.org/wiki/Delta_method) That is if you feel ok with the assumption of asymptotic normality of your $\vec p$ estimator. $\endgroup$ – Chap Jun 17 '12 at 2:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.