Skip to main content
added 547 characters in body
Source Link
A. Donda
  • 3.2k
  • 17
  • 36

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation testtest*: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

*) See Good, Permutation, Parametric and Bootstrap Tests of Hypotheses, 3rd ed., Springer 2005, section 3.2.1. The procedure can be traced back at least to Fisher, The Design of Experiments, Oliver & Boyd 1935, section 21, where he describes an alternative to the paired $t$-test that drops the assumption of normality and tests whether the paired samples come from the same distribution. As whuber pointed out, for the one-sample test the corresponding assumption is that the distribution is symmetric around 0 under the null hypothesis.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test*: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

*) See Good, Permutation, Parametric and Bootstrap Tests of Hypotheses, 3rd ed., Springer 2005, section 3.2.1. The procedure can be traced back at least to Fisher, The Design of Experiments, Oliver & Boyd 1935, section 21, where he describes an alternative to the paired $t$-test that drops the assumption of normality and tests whether the paired samples come from the same distribution. As whuber pointed out, for the one-sample test the corresponding assumption is that the distribution is symmetric around 0 under the null hypothesis.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

added 555 characters in body
Source Link
A. Donda
  • 3.2k
  • 17
  • 36

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.


Update after the poster's comments: $t$-test vs 0.8 gives $p = 0.27$, clearly non-significant, so the question whether the test is correct here is not really relevant. Sign-permutation test vs 0.8 gives $p = 0.30$, again a decent agreement, which indicates that the $t$-test isn't too bad.

Generally I'd recommend here to just use the sign-permutation test. If you have more data, you have many more permutations, which means you cannot generate them all. In this case, use a randomly drawn subset of the permutations (a.k.a. "Monte Carlo").

added 18 characters in body
Source Link
A. Donda
  • 3.2k
  • 17
  • 36

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform thea one-sided one-sample $t$-test on these 24 numbers, the result is $p = 1.58 \cdot 10^{-5}$$p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform the $t$-test on these 24 numbers, the result is $p = 1.58 \cdot 10^{-5}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

If the one-sample $t$-test you're looking at is vs 0, the fact that 15 of your 24 numbers are positive and the rest is not negative but 0 is already a quite strong indication that the true mean of the distribution your samples are from is larger than 0.

If you perform a one-sided one-sample $t$-test on these 24 numbers, the result is $p = 7.9 \cdot 10^{-6}$, far below the common significance level of 0.05. So even if the $t$-test is not exact in this case, it is quite unlikely that a correct test would give you a non-significant result.

A non-parametric alternative to the one-sample $t$-test is the sign-permutation test: Compute the mean of the numbers, but also on numbers where the signs have been switched (+ to -, - to +). The $p$-value is then the fraction of permutation means which are larger than or equal to the actual mean. There are $2^{24} = 16777216$ such permutations. The result is $p = 3.05 \cdot 10^{-5}$. Not only is this a significant result, but it also agrees well with the result of the $t$-test, which indicates any violation of the normality assumption here is not very strong.

Source Link
A. Donda
  • 3.2k
  • 17
  • 36
Loading