0
$\begingroup$

Let's say that we have 2 data samples $A$ and $B$, containing $N = 10000$ samples each, which are generated by two random variables $X_{A}$ and $X_B$. If I apply the student's T-test and apply a value below 0.01, can I formulate the following statement?

Data samples $A$ and $B$ are significantly different by a threshold of $0.01$.

Otherwise, how can I correctly interpret this with a statement?

$\endgroup$
2
  • 1
    $\begingroup$ Are you familiar with t-statistics and p-value? From the question alone it appears these two concepts were mixed up. I'd encourage you to read some introduction on t-test like this one, and perhaps revise the question by adding a general statement on what you'd like to show. If you have ran the test, feel free to post the results as well. $\endgroup$ Commented Jul 28, 2016 at 18:44
  • $\begingroup$ How about now ? $\endgroup$
    – Simon
    Commented Jul 28, 2016 at 18:58

2 Answers 2

2
$\begingroup$

It's just a very conventional answer and if anything is unclear please feel free to follow up. Usually, when reporting results of t-test, the statement should include the t-statistics and p-value. For instance (data are made up):

"The mean of $X_A$ is significantly higher/lower than that of $X_B$ (t = 3.36, p = 0.0004)"

Whether the difference is statistically significant or not depends on the p-value and false positive rate (aka type I error rate). For test that have a type I error rate of 5%, we declare the difference is significant if the p-value is lower than 0.05.

Meanwhile, t-statistics is a summary of the ratio of mean difference to the variances (You can roughly conceptualize this as the signal to noise ratio.) For that statistics, the bigger it is, the lower will be the p-value; and a significant difference is more likely.

$\endgroup$
0
$\begingroup$

Because the difference of the means of A and B is significant chances are high, that Xa and Xb have different means. If the p-value is small, your sample is sufficiently unlikely in the case of equal means in Xa and Xb. Therefore it is generally accepted, to conclude, that the means in Xa and Xb are different.

$\endgroup$
1
  • 1
    $\begingroup$ Unless you are working within a Bayesian framework, you cannot move from the significance of the test (p-value) to the chances that the variables have different means. $\endgroup$ Commented Jul 28, 2016 at 19:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.