What is the correct way of accepting the null hypothesis in a coin flip simulation program?

Question

I am new to statistics so please bear with my question. There are some similar questions to mine but I didn't get the clear answer after reading them. I have a program that simulates the coin tossing situation. The user enters a guess and the program determines whether the user's guess is correct or not. Each user's guess will be considered as the null hypothesis. For example the user enters: the number of getting heads in 1000 coin flips is 400 if the probability of each single flip is 0.5. Then based on calculating the lower and upper critical values with the significance value of 0.05, the program informs the user whether their guess is correct or not. To code my program, I can think of two different ways:

• (1) The program receives the number of flips from the user which is 1000 in this example and then does the 1000 flips for 100 separate times and when all experiments are done, the program calculates the mean of heads:

  

  where each a(i) is the number of heads in each 1000 flips. Eventually if the calculated mean is between lower and upper critical values, the user's guess is accepted, otherwise it is rejected.

• (2) We declare a variable t. Then the program repeats the 1000 flips experiment for 100 separate times, after each 1000 flips, if the number of heads is between the lower and upper critical values, the value of t is incremented by one. After all experiments are done, if the value of t is greater than 95 we accept the user's guess else we don't. (95 is the result of the number of experiments which is 100 multiplied by the significance value 0.05)

Now I wonder which of the above is logically correct to be safe from the Type 1 and Type 2 decision errors of hypothesis testing?

Answer 1

In hypothesis testing with any test statistic, we have a null hypothesis and an alternative hypothesis. Given a p-value for the test statistic, selected in advance and conventionally 0.05, we reject the null hypothesis if p < 0.05 and therefore accept the alternative hypothesis.

This is clearly an application of the law of the excluded middle. Both the null and the alternative cannot be true at the same time. The twist in statistics is that the conclusion is subject to an uncertainty arising out of random variation, which is why we have confidence intervals. Thus, when it's said that such-and-such disproves x, we really mean that it fails to disprove x.

In the example, of a user's guess, the null hypothesis is better conceptualized as the user's correct guesses are not different from random and the null hypothesis that the user's predictions are more accurate than random guesses.

A type I error is a false positive, rejecting the null when it is in fact true. A type II, false negative is failing to reject the null when it is in fact false.

It's a choice of evils. You can reduce type I errors by choosing a more stringent p-value, say 0.01, instead of 0.05, but that increases the likelihood of type II errors, which can be mitigated by a larger sample, but not eliminated.