how to calculate p-value based on # of trials and observed values?

Question

In this article, the p-value is calculated as 0.32 for observing 55 heads in 100 coin flips.

https://netflixtechblog.com/interpreting-a-b-test-results-false-positives-and-statistical-significance-c1522d0db27a

"In the example of observing 55 heads in 100 coin flips, we calculated a p-value of 0.32. Because the p-value is larger than the 0.05 significance level, we conclude that there is not statistically significant evidence that the coin is unfair."

However, when i try this using scipy library, here's what i get:

from scipy import stats

n = 100  # Number of coin flips
p = 0.5  # Probability of getting heads with a fair coin
observed_heads = 55  # Observed number of heads

# Calculate the p-value
p_value = 1 - stats.binom.cdf(observed_heads - 1, n, p)
print("P-value:", p_value)

P-value: 0.18410080866334777

How do they calculate p-value of 0.32?

Answer 1

You are conducting a one-sided test, which discounts $\le 45$ as evidence against the null of fairness. This is the appropriate hypothesis for testing that there is a bias in favor of heads.

The Netflixers are doing a two-sided test, so seeing too few heads is also in the direction of this alternative hypothesis. This is a test of fairness, so it will be sensitive to a bias in either direction:

. bitesti 100 55 0.5, detail

Binomial probability test

            N   Observed k   Expected k   Assumed p   Observed p
----------------------------------------------------------------
          100           55           50     0.50000      0.55000

  Pr(k >= 55)            = 0.184101  (one-sided test)
  Pr(k <= 55)            = 0.864373  (one-sided test)
  Pr(k <= 45 or k >= 55) = 0.368202  (two-sided test)

  Pr(k == 55)            = 0.048474  (observed)
  Pr(k == 46)            = 0.057958
  Pr(k == 45)            = 0.048474  (opposite extreme)

Generally, outcomes with $k$ successes are considered “as extreme or more extreme” than the observed outcome of 55 if $\Pr(k) \le   \Pr(55) = 0.048474$.

The difference between 0.368202 and 0.32 is most likely due to using an approximate t-test of means instead of the exact binomial, which gives a p-value of $\approx 0.32$.