I have the regular hypothesis where I'm testing whether the mean of one variable is larger than the mean of the other variable or not:
H0: mean(variable1) <= mean(variable2) H1: mean(variable1) > mean(variable2)
I've ran the Welch's test variation of the student t test in R and my output is:
Welch Two Sample t-test data: x and y t = -2.8207, df = 43.367, p-value = 0.9964 alternative hypothesis: true difference in means is greater than 0 95 percent confidence interval: -0.08806587 Inf sample estimates: mean of x mean of y 0.6708163 0.7260000
The significance level I'm using is 5%. So based on the output, I know I can't reject the H0 hypothesis on a 5% significance level. Since the difference in mean from x and y is about -0.05 and is within the 95% CI, I can yet again conclude that I can't reject the H0 hypothesis.
Subquestion: Does analyzing both the result with the CI and the p-value somehow empower my statement? Can I be 'more confident' that H0 can or cannot be rejected if I analyze the CI if my p-value already indicates it's above or below the significance level? I assume a contradiction between p-value testing and CI testing cannot occur.
Main question: how should I interpret the t-value exactly? From what I learned, the 'further' the t-value strays from 0, the more likely that the effect is 'statistically significant'. First of all, I'm not sure what it means exactly when they say an effect is statistically significant. I'm guessing it means our test is more likely to be accurate? And how far from 0 would be safe and why?