I am from a biology background. Using t, χ2, F tests day-to-day, following like a recipe. However, I feel I must understand the background of this.

I took an online lecture on p-value and hypothesis testing.

if your p-value is less than 0.01, then you have a very strong case against null hypothesis very strong case. The meaning of this is only in 1 % of the cases your null hypothesis will hold true in 99 % of the cases your null hypothesis will be false

Without many mathematical notations, could someone help me to understand;

1. why having a low probability is strong?

2. What p-value really tells in hypothesis testing? because some say it'd not about probability.

3. Is setting a null hypothesis a mathematical requirement?

4. Is normal distribution a hypothetical thing? because why we have to use different t, χ2, F distributions for different data types to calculate p-value.

I am not sure why I fail every time to understand this; is it mathematics or is it the language (wording) use to explain these concepts.

  • 3
    $\begingroup$ The quote you have put up is an outright wrong interpretation, but unfortunately is very common, which is probably why you may be confused. I hope to expand it to a full answer when I find the time, but will be grateful if others can provide their input at the meantime. $\endgroup$
    – B.Liu
    Mar 6 '21 at 13:10
  • $\begingroup$ There are several reasons to close this post: it comprises several distinct questions (and so needs to be more focused) and all of them have been answered (extensively) in other threads. I have chosen a duplicate that might help you the most in making progress with this set of questions. $\endgroup$
    – whuber
    Mar 6 '21 at 17:17
  • $\begingroup$ Hi Whuber, Could you re open this question for me. I could not find easily digestible answer. My background is not in mathematics at all. I hope someone could kindly help me to understand this. Thank you $\endgroup$
    – Dendrobium
    Mar 31 '21 at 12:22
  • 1
    $\begingroup$ @Dendrobium There are at least 7 highly-voted answers in the question whuber used in their close vote (I recommend getting a drink and reading the answer from whuber themself). Please can you edit your question and let the community know where exactly do you get stuck and possibly help? It also has the side effect of putting the question to be considered for reopening. $\endgroup$
    – B.Liu
    Mar 31 '21 at 13:27
  • $\begingroup$ @B.Liu I have edited the question. I am looking for a non-mathematical answer if possible. Thank you for your kind support. $\endgroup$
    – Dendrobium
    Apr 5 '21 at 13:20