1
$\begingroup$

Question:

In general, given a statistic with a highly non-normal (but known) pdf/cdf, how can one use the known pdf/cdf to compute/approximate confidence intervals for the statistic? What options are available? After some reading, one potential (I think) avenue for doing this is via data transformations using copulas? Is this a viable option? I am open to any and all useful techniques.


Reason for asking (specific use case):

I have the following statistic for which I have closed-form expressions for the pdf and cdf: \begin{equation} G=\frac{\bar{X}-\bar{Y}}{\hat{X}-\hat{Y}}\,, \end{equation} where \begin{align} \bar{X}&=\frac{1}{n}\sum_{i} X_{i}\,,\\ \bar{Y}&=\frac{1}{m}\sum_{i} Y_{i}\,,\\ \hat{X}&=\frac{1}{n-1}\sum_{i} (X_{i}-\bar{X})^{2}\,,\\ \hat{Y}&=\frac{1}{m-1}\sum_{i} (Y_{i}-\bar{Y})^{2}\,,\\ \end{align} for $X_{i}\sim\mathcal{N}(\mu_{X},\sigma_{X}^{2})$ and $Y_{i}\sim\mathcal{N}(\mu_{Y},\sigma_{Y}^{2})$. As one can see, this statistics is based off of two independent samples of different sample sizes where the observations of each sample are i.i.d. normal. The density of $G$, $f_{G}(g)$, can be heavily skewed (even bimodal) depending on the parameters used. I want to compute confidence intervals for $G$ when the distribution is heavily skewed. Also, the moments of $f_{G}(g)$ do not exist; however, I have approximations given $\mathrm{Pr}(\hat{X}-\hat{Y}>0)\approx 1$. Of most interest to me is CI's for $G$ when positive support of $\hat{X}-\hat{Y}$ can be assumed.

$\endgroup$
  • 1
    $\begingroup$ Could you explain the meaning of $\hat X-\hat Y$ in the denominator? $G$ is a very strange statistic; it looks more like an erroneous version of a $t$ statistic. Moreover, it will never be mathematically true that $\hat X - \hat Y$ has positive support, making $G$ even more suspect. $\endgroup$ – whuber Jul 13 '17 at 18:37
  • $\begingroup$ @whuber The meaning of $\hat{X}-\hat{Y}$? It is simply the difference of the sample variances from the two independent samples. $G$ is a statistic that has significant use in electro-optics. And yes I understand that it will never have positive support. That said, when the probability of the denominator being negative is very small, Taylor expansion can be used to approximate the "mean" and "variance" of the pdf. $\endgroup$ – Aaron Hendrickson Jul 13 '17 at 18:40
  • $\begingroup$ What I'm asking about is what a difference in variances might mean. It looks like it results from some conceptual or computational mistake or perhaps an inappropriate model formulation. If that is so, then a much better solution to your problem would be to fix that mistake rather than analyze this statistic. $\endgroup$ – whuber Jul 13 '17 at 18:50
  • $\begingroup$ @whuber The differences in variances on it's own carries little meaning. However, the ratio that defines $G$, is an estimate for the conversion gain of an image sensor. What that means is that it estimates the number of photons required to produce an output of one unit (such as volts or digital numbers). If your curious, $G$ is called the "Photon Transfer conversion gain". I recently derived it's pdf and cdf. Trying to use it for calculating CI's of $G$. $\endgroup$ – Aaron Hendrickson Jul 13 '17 at 18:57
  • 1
    $\begingroup$ The problem with testing distributions without moments by using sample data is that it can fail to uncover the risk of making a huge error: things will look fine for a long time and then all of a sudden a whopping big number comes in that screws everything up. That might be something to think about. The transformation is a nice idea, but how do you find it? Since you don't know the underlying parameters, you know $F_G$ only with (great) uncertainty. There are ways to overcome that: the BCa bootstrap CI is one notable example that implicitly estimates and applies that transformation. $\endgroup$ – whuber Jul 13 '17 at 20:08
1
$\begingroup$

I would use a resampling algorithm (e.g. bootstrapping). After taking 2000 samples, I would just calc the 2.5% and 97.5% intervals of the statistics. If you want to lower the expected error, just take 20 000 samples.

$\endgroup$
  • $\begingroup$ The extreme skewness of $G$ suggests this approach is unlikely to give good intervals. At the least they ought to be corrected for the skewness. Although it's always nice to have more data, recommending that someone take huge numbers of samples is rarely an acceptable or practicable solution. At the least you would have to have a very good reason to support that recommendation. $\endgroup$ – whuber Jul 13 '17 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.