I'm conducting a simulation study where I repeatedly sample heights from a normal distribution with a mean of 175 cm and standard deviation of 6 cm. I then calculate a one-sample t-test p-value for each sample against a null hypothesis mean of -175 cm. My sample sizes are quite large (200 observations per sample).
Theoretically, I expected the p-values to follow a unimodal distribution since i thought comparing it to a small sample that have a uniform distribution,so we'll have more population and the mean will be more accurate and close to 175. Which will make the p-value closer to range 1. (skewed to the left). but I was wondering if a larger sample size might change this distribution. Specifically, I used this code:
iterations = 5000 p_values = np.empty(iterations) for i in range(iterations): sample_heights = np.random.normal(175, 6, 200) p_values[i] = p_value_calculator(sample_heights, 200, 175) plt.hist(p_values, bins=30, edgecolor='black') plt.xlabel('p-value') plt.ylabel('Count') plt.title('Distribution of p-values')
Despite having a large sample size, I noticed that the distribution of p-values appears uniform, which was initially surprising to me. Does sample size affect the distribution of p-values or is it always uniform when the null hypothesis is true?"