1
$\begingroup$

If I'm running a simple 1-sample t-test, and I have x-bar, s, n, and mu, where p-hat and s come from the following type of numbers:

0
0
0
0
0
0
0
0
0
.8
.8
.8
.8
1.2
1.2
1.2
1.6
1.6
1.6
2
2
2
2
2

I.e, from only 5 different possible "scores" on a "test", can I proceed as usual with the t-test? Is there a different procedure for data like these?

$\endgroup$
7
  • $\begingroup$ The $t$-test assumes normality, at least for smaller samples, and these data are markedly non-normal. You would probably want to use a non-parametric test, but more importantly, what is the actual question you're trying to answer? $\endgroup$
    – dsaxton
    Commented Jul 29, 2015 at 1:24
  • 2
    $\begingroup$ @dsaxton The sampling distribution of the mean will nevertheless be so close to normal that one might proceed to use a t-test with substantial confidence that it will be accurate. $\endgroup$
    – whuber
    Commented Jul 29, 2015 at 1:33
  • $\begingroup$ @dsaxton Would prefer not to disclose the actual purpose of the test, but know that I have many more items in the sample (more than 30) so the central limit theorem should apply, no? I don't think normality is the issue. I'm not sure there is an issue; I'm just not sure that there isn't, either. $\endgroup$
    – japem
    Commented Jul 29, 2015 at 1:41
  • $\begingroup$ @whuber, but isn't the problem that the CLT doesn't tell us whether the estimated variance(s) will be $\chi^2$-distributed. $\endgroup$
    – A. Donda
    Commented Jul 29, 2015 at 2:05
  • $\begingroup$ @A.Donda That's a good point, but it's not directly relevant. What matters is whether the Student t distribution is a good enough approximation to the $t$ statistic to give reliable p-values. Even for small amounts of binomial data--which can be highly skewed--it can be a good approximation for p-values near conventional test sizes of 1 to 10 percent. This is an assertion based on experience, not the CLT. Its applicability in any particular dataset can be tested by looking at the bootstrap distribution of the t statistic. $\endgroup$
    – whuber
    Commented Jul 29, 2015 at 11:39

2 Answers 2

1
$\begingroup$

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test*: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

*) See Good, Permutation, Parametric and Bootstrap Tests of Hypotheses, 3rd ed., Springer 2005, section 3.2.1. The procedure can be traced back at least to Fisher, The Design of Experiments, Oliver & Boyd 1935, section 21, where he describes an alternative to the paired $t$-test that drops the assumption of normality and tests whether the paired samples come from the same distribution. As whuber pointed out, for the one-sample test the corresponding assumption is that the distribution is symmetric around 0 under the null hypothesis.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

$\endgroup$
12
  • $\begingroup$ It's not vs 0. It's vs something closer than that to the sample mean. I fudged these numbers a bit, but in the actual dataset I'm testing them vs about .36. So imagine about .8 in this dataset (x-bar is .9) $\endgroup$
    – japem
    Commented Jul 29, 2015 at 2:28
  • $\begingroup$ @japem why would you test vs different values depending on the size of the data set? Your null hypothesis should be specified independent of the data, including the sample size. You're not deriving these values somehow from the data?! $\endgroup$
    – A. Donda
    Commented Jul 29, 2015 at 2:37
  • $\begingroup$ I think you may have misunderstood me... the numbers I provided in the original example are different from my actual data because I don't want to divulge those. My real null hypothesis is about .36. For this problem, just imagine that my null hypothesis is .8. $\endgroup$
    – japem
    Commented Jul 29, 2015 at 2:42
  • 1
    $\begingroup$ @japem, still, how do you get these null hypothesis numbers? And how do you expect us to reasonably answer your question whether you can apply a $t$-test to the data, if you're showing us other data which might have completely different properties? $\endgroup$
    – A. Donda
    Commented Jul 29, 2015 at 2:43
  • $\begingroup$ The average of other data. I'm testing a specific person against the average. $\endgroup$
    – japem
    Commented Jul 29, 2015 at 2:44
0
$\begingroup$

I would be quite comfortable that using a t-test here especially since you have reported "many more items" in the sample. The Central Limit Theorem dictates that the mean will be very close to normally distributed and thus the t-test will be appropriate.

$\endgroup$
1
  • 1
    $\begingroup$ You might also consider a non-parameteric approach too and compare the two results. For large samples, you'll often find the two results agree. $\endgroup$ Commented Jul 29, 2015 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.