How to use ttest and reject null hypothesis

I'm using ttest for two populations, I know that we can reject the null hypothesis when the p-value is less that 0.1 (with 90% confidence). but does it make sense to get a pvalue of say .17 and then say the two means are different with 83% confidence?

A p-value is used to assess the strength of the evidence against the Null hypothesis. A smaller p-value corresponds to stronger evidence against the Null hypothesis.

Let me clarify some details regarding Confidence: Confidence is not 1-p-value. Confidence is 1-Alpha, where alpha is a predetermined threshold to determine if a difference is significant.

Let's suppose we choose an Alpha = 0.05. If we perform the test and get a p-value=0.001, we can conclude: "There is sufficient evidence to reject the Null hypothesis". Confidence doesn't really mean anything when interpreting the results of the test, and it certainly does not mean that confidence is 1-0.001=0.999.

If you have a p-value of 0.17, this does not mean you have 83% confidence in the result. It simply means there is a 17% chance of obtaining the sample results assuming the Null hypothesis is true.

Confidence is less involved when discussing hypothesis testing, but more meaningful when discussing a confidence interval. Let me know if this helps answer your question. I'm happy to help further if needed.

• Thanks, so when performing a ttest can I set a confidence level then?
– HHH
Mar 4 '19 at 18:26
• No, I wouldn't call it a confidence level. We don't have a level of confidence in the p-value; the p-value is what it is, with no uncertainty. When performing a hypothesis test, you set the threshold for concluding significance, say at 0.05 or 0.1. If the p-value is less than the threshold, then you reject the Null hypothesis. You cannot make a statement like "I am 95% confident the null hypothesis is false". The validity of the null hypothesis is judged by the p-value primarily.
– JLG
Mar 4 '19 at 21:26