2
$\begingroup$

Question:

In general, given a statistic with a highly non-normal (but known) pdf/cdf, how can one use the known pdf/cdf to compute/approximate confidence intervals for the statistic? What options are available? After some reading, one potential (I think) avenue for doing this is via data transformations using copulas? Is this a viable option? I am open to any and all useful techniques.

Reason for asking (specific use case):

I have the following statistic for which I have closed-form expressions for the pdf and cdf: \begin{equation} G=\frac{\bar{X}-\bar{Y}}{\hat{X}-\hat{Y}}\,, \end{equation} where \begin{align} \bar{X}&=\frac{1}{n}\sum_{i} X_{i}\,,\\ \bar{Y}&=\frac{1}{m}\sum_{i} Y_{i}\,,\\ \hat{X}&=\frac{1}{n-1}\sum_{i} (X_{i}-\bar{X})^{2}\,,\\ \hat{Y}&=\frac{1}{m-1}\sum_{i} (Y_{i}-\bar{Y})^{2}\,,\\ \end{align} for $X_{i}\sim\mathcal{N}(\mu_{X},\sigma_{X}^{2})$ and $Y_{i}\sim\mathcal{N}(\mu_{Y},\sigma_{Y}^{2})$. As one can see, this statistics is based off of two independent samples of different sample sizes where the observations of each sample are i.i.d. normal. The density of $G$, $f_{G}(g)$, can be heavily skewed (even bimodal) depending on the parameters used. I want to compute confidence intervals for $G$ when the distribution is heavily skewed. Also, the moments of $f_{G}(g)$ do not exist; however, I have approximations given $\mathrm{Pr}(\hat{X}-\hat{Y}>0)\approx 1$. Of most interest to me is CI's for $G$ when positive support of $\hat{X}-\hat{Y}$ can be assumed.

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ Could you explain the meaning of $\hat X-\hat Y$ in the denominator? $G$ is a very strange statistic; it looks more like an erroneous version of a $t$ statistic. Moreover, it will never be mathematically true that $\hat X - \hat Y$ has positive support, making $G$ even more suspect. $\endgroup$
    – whuber ♦
    Commented Jul 13, 2017 at 18:37
  • $\begingroup$ @whuber The meaning of $\hat{X}-\hat{Y}$? It is simply the difference of the sample variances from the two independent samples. $G$ is a statistic that has significant use in electro-optics. And yes I understand that it will never have positive support. That said, when the probability of the denominator being negative is very small, Taylor expansion can be used to approximate the "mean" and "variance" of the pdf. $\endgroup$
    – Aaron Hendrickson
    Commented Jul 13, 2017 at 18:40
  • $\begingroup$ What I'm asking about is what a difference in variances might mean. It looks like it results from some conceptual or computational mistake or perhaps an inappropriate model formulation. If that is so, then a much better solution to your problem would be to fix that mistake rather than analyze this statistic. $\endgroup$
    – whuber ♦
    Commented Jul 13, 2017 at 18:50
  • $\begingroup$ @whuber The differences in variances on it's own carries little meaning. However, the ratio that defines $G$, is an estimate for the conversion gain of an image sensor. What that means is that it estimates the number of photons required to produce an output of one unit (such as volts or digital numbers). If your curious, $G$ is called the "Photon Transfer conversion gain". I recently derived it's pdf and cdf. Trying to use it for calculating CI's of $G$. $\endgroup$
    – Aaron Hendrickson
    Commented Jul 13, 2017 at 18:57
  • 1
    $\begingroup$ The problem with testing distributions without moments by using sample data is that it can fail to uncover the risk of making a huge error: things will look fine for a long time and then all of a sudden a whopping big number comes in that screws everything up. That might be something to think about. The transformation is a nice idea, but how do you find it? Since you don't know the underlying parameters, you know $F_G$ only with (great) uncertainty. There are ways to overcome that: the BCa bootstrap CI is one notable example that implicitly estimates and applies that transformation. $\endgroup$
    – whuber ♦
    Commented Jul 13, 2017 at 20:08

1 Answer 1

Reset to default
1
$\begingroup$

I would use a resampling algorithm (e.g. bootstrapping). After taking 2000 samples, I would just calc the 2.5% and 97.5% intervals of the statistics. If you want to lower the expected error, just take 20 000 samples.

Improve this answer
$\endgroup$
2
  • $\begingroup$ The extreme skewness of $G$ suggests this approach is unlikely to give good intervals. At the least they ought to be corrected for the skewness. Although it's always nice to have more data, recommending that someone take huge numbers of samples is rarely an acceptable or practicable solution. At the least you would have to have a very good reason to support that recommendation. $\endgroup$
    – whuber ♦
    Commented Jul 13, 2017 at 18:39
  • $\begingroup$ Also, I believe that resampling will estimate the sampling distribution. The confidence intervals are defined as an interval containing a parameter whose value defines the sampling distribution. Thus, you might be looking for the predictive distribution (as opposed to the sampling distribution). That migh be obtained from pivotal of the variables. $\endgroup$
    – Denis Cousineau
    Commented Aug 6, 2022 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.