0
$\begingroup$

My Null for the T-test is

h0: -tcritical < Tstat < +tcritical

I would like a confidence level of 95%.

If my empirical result satisfies the null, but not my confidence requirements (the p value was 0.36),

does this mean that the test is inconclusive? Or that the null is not-rejected/rejected?

The t-test I have performed is the 'T-test:unequal variances' on Microsoft Excel

Identical procedure to this link

I'm a beginner with modelling, so I thank you for your patience in explaining things

$\endgroup$

marked as duplicate by gung, whuber Jan 5 '15 at 16:13

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ What do you mean by "my empirical result satisfies the null"? $\endgroup$ – Aksakal Jan 5 '15 at 15:02
  • $\begingroup$ My empirical result satisfies the requirements of H0. I.e the result calculated X fits between the required range. -tcritical < X < +tcritical $\endgroup$ – Harry Jan 5 '15 at 15:22
  • $\begingroup$ What does it take to satisfy your requirements? $\endgroup$ – Aksakal Jan 5 '15 at 15:25
  • $\begingroup$ If the T-statistic generated is greater than '-tcritical two-tail' but less than '+tcritical two-tail', then I believe there is no significant difference between the means of my two samples. The exact procedure I have used is described better here link. The main difference is I've tried to acknowledge p-values (I'm not sure if I can?) $\endgroup$ – Harry Jan 5 '15 at 15:29
0
$\begingroup$

The obtained p-value 0.36 is greater than 0.05, therefore you can not reject $H_0$ at 95% confidence. The null is that the means are equal, so your test is telling is that there's no evidence to say they are not equal. I would not call this inconclusive, because this is the best you can get from statistical inference.

If your p-value was 0.049 or 0.051, then formally you're supposed to take a binary decision to reject or not, but I'd be worried about this value. The reason is that any little change in the experiment could push the value up or down. Thus, I'd call this inconclusive but not in the statistical sense, but rather for my research design to maybe conduct more experiments or collect more data and such.

$\endgroup$
1
$\begingroup$

In a classical statistics framework we say that you have "failed to reject the null". In other words, there is not enough evidence to say that the null is false. This is not the same as saying the null is true.

Now, you may be asking, "if there is not enough evidence to say that the null is false and we are also not saying the null is true - then what are we saying?". The answer is that we are not saying anything. This can be very unsatisfying to people and is a big criticism of the classical statistics framework.

A simple alternative that I recommend is to look at confidence intervals rather than hypothesis test results. A confidence interval gives you a reasonable range of values that could be the true value of what you are measuring whereas a hypothesis test just gives you a binary result: is there enough evidence to disbelief the null or not?

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.