# Can we change the estimated population mean in Hypothesis Testing?

My goal is to learn hypothesis testing. My understanding is that we do not have the population data. Therefore, we do not know the estimated population mean. So, we guess the population mean. If that so, can we change the population mean until the p-value is below 0.05, so we have enough evidence to reject the null hypothesis?

The code

import pandas as pd
from scipy import stats

sample = pd.Series([1,2,3,4,5,6,7,8,9,10])

# Hypothesis Testing:
#
# H0: μ <= 3.5 (previously it was μ <= 5.5,
#               then 5, then 4.5, I change it until I get p value less than 0.05)
# H1: μ > 3.5

t_stat, p_value = stats.ttest_1samp(a = sample, popmean=3.5, alternative="greater")

print("sample mean:", sample.mean())
print("t-statistic: {0:.2f} p-value: {1:.2f}".format(t_stat, p_value))

print("since we do not have the population data, "
"popmean=3.5 is not meaningful, "
"we can change it until we get enough evidence to reject the null hypothesis")

print("critical value: 5%, therefore confidence interval: 90%")

print("because p-value is below 0.05, we have enough evidence to reject the null hypothesis. "
"therefore, we can conclude that the mean population is more than 3.5")

print("population standard deviation {0:.2f}".format(sample.std(ddof=0)))

print("if we imagine the sample as the weight of waste, the conclusion is, "
"90% of population data is between population mean + 3 * population standard deviation, "
"in this case 3.5 +- 3 * 2.87")
$$$$

• The population parameter is fixed a priori and is something we want to learn from data. Commented Oct 18, 2022 at 4:39
• @utobi therefore, we can't change the popmean arg as we like? e.g stats.ttest_1samp(a = sample, popmean=5.5, alternative="greater") the result is pvalue is larger than 5%, so change it to popmean=5, the pvalue is still larger than 5%, and so on? Commented Oct 18, 2022 at 4:53
• Do you mean to change the null hypothesis from testing 0 to some other value? Commented Oct 18, 2022 at 5:02
• @rep_ho yes, changing the H0: μ <= 5.5 to H0: μ <= 5, the sample data remain the same. Commented Oct 18, 2022 at 5:27
• You can change hypothesis, but that has to be done BEFORE observing the data. Commented Oct 18, 2022 at 7:17

Expanding a bit over my comments to the post and in addition to @Christian Henning's answer, there is nothing special about hypothesis testing. Indeed, the rationale behind hypothesis testing and confidence intervals follows that of the Scientific Method, which we have been exposed to since middle school.

It is the usual cycle

Formulate a hypothesis -> run experiments and collect data -> analyse data and look if your hypothesis is supported.

The data collected may not support your hypothesis and that's perfectly acceptable. In that case, there are two things happening:

1. the hypothesis is true but the data are not enough or
2. the hypothesis is actually not true.

If you do not get evidence about your hypothesis, you can either decide to collect new data or rethink the hypothesis...

Nevertheless, tailoring the scientific hypothesis to what the data say is scientifically the most terrible thing you could do, since

• Observed data are subject to randomness, i.e. your results may not be replicable.
• Data should and can only offer you evidence about a scientific theory, but they cannot tell WHY or IF the theory is TRUE. Truthiness has to be based on domain-specific arguments.

Thus, to sum up, you can make the data tell you what you want, but what you will get will be, at best, useless or harmful.

• I believe people that have similar problem should: 1. read @Christian Henning's answer to understand the terrible idea of "changing the estimated popmean", @utobi's Answer to understand what to do next if you fail to reject the H0: μ <= estimated popmean and @monkeyunited's Answer on reddit r/datascience to understand the logics on when to use hypothesis testing. Commented Oct 19, 2022 at 2:29
• Anyway, let me recap, do you mean If you fail to reject H0, you should gather more data or just stop there because perhapse the alternative hypothesis is actually not true and not the null hypothesis e.g H0: μ <= estimated popmean, the estimated popmean should stay the same. Commented Oct 19, 2022 at 2:31
• @kidfrom more or less. $H_0\mu\leq\bar x$ is not useful, since you do not know $\bar x$ beforehand, instead, you should have $H_0\mu\leq \mu_0$, for some $\mu_0$ coming from theory. Commented Oct 19, 2022 at 8:55

The idea of hypothesis testing is that you are interested in testing a particular hypothesis before having seen the data. If you change the hypothesis based on the data, the data will not address the hypothesis you were interested in before.

There is also another problem, which is that the theory behind the hypothesis tests assumes that the hypothesis is not chosen based on the same data on which it is tested. This means that testing a hypothesis chosen based on the data is invalid.

• Do you mean by the hypothesis is not chosen based on the same data on which it is tested that first data analyst did a population height survey last year and gather enough data to call it as the population height mean. Then, the second analyst use that population height mean to do another hypothesis testing, e.g average height of people that had exposure to lead poisoning and gather the data specifically people that have exposure to lead poisoning. Let's say the second analyst use One Sample Two Sided with H0: μ <= mean from the first data analyst`. Is that the correct way to do it? Commented Oct 19, 2022 at 2:41
• @kidfrom I don't understand this question. Unless you know the whole population, you cannot "gather enough data to call it as the population height mean". Also I don't get why anybody would be interested in hypothesis testing at all. Isn't this about just estimating the population mean? You may be interested in having a confidence interval, but why would you be interested in testing any specific value? (If in a general situation a second scientist is interested in testing what a first scientist claims, the thing is they shouldn't use the same sample as was used by the first one to test this.) Commented Oct 19, 2022 at 10:05