0
$\begingroup$

I have two distributions relative to two experimental conditions.

I compute a certain index (i.e. coherence) describing each distribution.

I want to see if there is a significant difference between the indexes of the two distributions.

But now I have one value per distribution.

Therefore I am using the jackknife approach to re-calculate two distributions of my indexes and check for statistical significance.

Q: Now i have two jackknife distributions, how can i actually see if they are significantly different?

  1. Can I just do a t-test between the two jackknife distributions?

  2. Do I have to "manipulate" my data before performing any test? the variance of my jackknife distributions is suspiciously low!

Thank you for your time.

Valerio

$\endgroup$
3
  • $\begingroup$ I think you may be looking for either a permutation test or bootstrapping but from your description I cannot be sure. $\endgroup$
    – mdewey
    Aug 28, 2017 at 12:35
  • $\begingroup$ Hi, thank you for your reply. In my case I am reluctant in getting data from other distributions (permutation?), and I don´t have enough samples to get random subsamples (bootstrapping?) of my dataset. I think the leave 1 out strategy is good enough to get a distribution back. $\endgroup$ Aug 28, 2017 at 12:45
  • $\begingroup$ the question is, what do i do with the output of the jackknife? $\endgroup$ Aug 28, 2017 at 12:47

1 Answer 1

0
$\begingroup$

You do not specify, but let's assume you used delete-1-jackknife.

Problem statemement

In that case, you have for each deletion $n$ unique Jackknife samples with the $i$th sample vector being $$X_i = \{X_1, X_2, X_3,..., X_n-1, X_n\}$$

You estimate your parameter with $ s(\cdot) $ which returns

$$ \hat\theta_i = s(X_i) $$

It seems to me that you are stuck with a vector $\hat\theta$ with $i$ entries. I assume you want to estimate the location parameters of two independent samples, and test whether they are significantly different?

To do so, you need to calculate the variance $V$ as $$V_{\theta} = \frac{n-1}{n}\sum_{i=1}^{n} (\hat\theta_i - \bar\theta)^2$$ with $$ \bar\theta = \frac{1}{n} \sum_{i=1}^n \hat\theta_i$$

Conclusion

To answer your two questions directly:

Can I just do a t-test between the two jackknife distributions?

No

Do I have to "manipulate" my data before performing any test? the variance of my jackknife distributions is suspiciously low!

Note the difference in the factor for estimating $V$. Instead of $\frac{1}{n}$, you use $\frac{n-1}{n}$. Imagine it as a correction factor to account for the repetition of samples within $X$.

Possible Solutions

After you have calculated $V_{\theta}$ and $\bar\theta$ from $X$, you can contrast it against the output from another independent sample, e.g. $V_{\beta}$ and $\bar\beta$ from $B$.

Your statistical test-value $Q$ could then be calculated as $$ Q = \frac{\bar\theta-\bar\beta}{\sqrt{V_{\theta}+V_{\beta}}}$$ You can derive your critical values for $Q$ from a normal distribution to reject $H_0$ if $Q \geq z_\alpha$ with $z_\alpha$ being the upper $\alpha$th percentile.

A sensible alternative is bootstrapping, as it allows you to estimate the confidence bounds of the difference directly without any recurse to a parametric distribution.

References

See also Hollander, Wolfe (1999): Nonparametric statistical methods. Wiley

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.