Does this NYT article incorrectly assume independent increments?

The article plots for every 100 women that use a certain type of contraception method the number of unplanned pregnancies over time.

https://www.nytimes.com/interactive/2014/09/14/sunday-review/unplanned-pregnancies.html?_r=0

In particular at the end of the article they say:

The numbers are calculated as follows:

$\mathbb P(\text{Not pregnant after year N}) = \mathbb P(\text{Not pregnant after year 1})^N$

Indeed, success rate of contraception is probability not pregnant in year 1. See e.g. https://www.cdc.gov/reproductivehealth/contraception/unintendedpregnancy/pdf/contraceptive_methods_508.pdf

This is true if probability pregnant in a year is independent of the year before, but seems highly unlikely to be true. If you use contraception the wrong way, it will probably go wrong in the first year, and if it does not, then it probably won't go wrong the year after?

• Not the first time such injustice is done, won't be the last time (there has already been > 2/12 years for additional injustices). Unwarranted independence assumption (often done in conjunction with premature expectation) is one of the most common, and severe analysis errors, not only by amateurs, but also by people educated in science, engineering, and even math, many even having PhDs. Throw in unwarranted Normality asumption and confusing P(B|A) with P(A|B), and you've covered the most common probability errors I see. I give some credit to author for disclosing formula. – Mark L. Stone Mar 26 '17 at 12:44
• Per capita rates such as are used in this article normalize otherwise widely disparate metrics. The author adheres to conventional practices. – Mike Hunter Mar 26 '17 at 12:57
• "The percentages indicate the number out of every 100 women who experienced an unintended pregnancy within the first year of typical use of each contraceptive method." is used as the succesrate – user103341 Mar 26 '17 at 13:02
• There must be empirical studies against which this can be verified. And this would also be a good question for skeptics.SE . – Olivier Mar 26 '17 at 14:14
• Some relevant reading: guttmacher.org/gpr/2007/05/… It notes that many women having repeat abortions were using "highly effective" methods of contraception when they got pregnant, and discusses systemic issues that lead to repeat unintended pregnancies, supporting your suspicion that independence is a bad assumption. Other answers have already discussed the implications of non-independence. – Geoffrey Brent Mar 27 '17 at 4:45

Sorry, I cannot agree with the independence assumption. Fertility in women, even without contraception is a function of age, such that, without contraception the

Chances of getting pregnant without IVF (in vitro fertilization)

Starting at about age 32, a woman’s chances of conceiving decrease gradually but significantly.
From age 35, the fertility decline speeds up.
By age 40, fertility has fallen by half.
At 30, the chance of conceiving each month is about 20%. At 40 it’s around 5%.
Note (mine) after age ~49 menopause occurs and when it does, women are infertile.


The rate of pregnancy is also a function of frequency of intercourse, which also changes with age:

About 5% of single women between the ages of 18 and 24 had sex 4 or more times per week, but 24% of married women did.
Like the men, just under half of the women between the ages of 25 and 59 had sex a few times per month to weekly, more than their single and partnered peers.
Sexual frequency did decrease with age for women, although almost a quarter of partnered women over age 70 had sex more than 4 times a week.


Relative time of ovulation, intercourse, and female age:

Finally, to consider the effectiveness of contraception on an annualized basis, one must consider not only decreasing fertility and variable but generally somewhat decreasing sexual frequency with age, but likely also myriad other factors. For example, the percentage of women who are postpartum increases with age, and postpartal women may have a different contraceptive usage effectiveness than the nulliparous, age of the partner at time of intercourse relative to ovulation, see image:

timing of intercourse relative to ovulation, having a huge impact on fertility, also reflects on the likelihood of pregnancy even when other factors, like contraception is considered. Thus, a woman who relies on the rhythm method, as well as one or more other methods of contraception, i.e., a woman who both knows her body functions, and uses that knowledge (and as knowledge is acquired) may eventually do an increasingly effective job of avoiding pregnancy, such that there is essentially no chance for independence of fertility with elapsed age.

Here is an account of the probabilities relevant to the problem at hand.

Denote by $z_n$ the event 'no pregnancy after $n$ years' for a woman using some type of contraception. Then $$P(z_N) = P(z_N | z_{N-1}) P(z_{N-1} | z_{N-2})\cdots P(z_2 | z_1)P(z_1).$$ The problem is that the NYT asumes $P(z_i | z_{i-1}) = P(z_1) = p$, for all $i$, while knowing $z_{i-1}$ may provide evidence that the woman makes good use of the contraception method and may be experienced with it. We should therefore expect that $$P(z_N | z_{N-1}) > P(z_{N-1} | z_{N-2}) > \cdots > P(z_1).$$ This implies $$P(\text{'at least 1 pregnancy after N years'}) < 1-p^n$$ rather than the equality claimed by the NYT.

Every woman $w$ using a particular type of contraceptive, among $M$ other woman, has her own fixed probability $q_w$ of not getting an unwanted pregnancy in a year. The average success rate of the contraceptive over one year is $p =\tfrac{1}{M}\sum_w q_w$. The average success rate after $N$ years, assuming year-to-year independence, is $\tfrac{1}{M}\sum_w q_w^N$. However, $$\tfrac{1}{M}\sum_w q_w^N \geq \left(\tfrac{1}{M}\sum_w q_w\right)^N = p^N,$$ by Jensen's inequality, with equality if and only if $q_w$ is constant over $w$.

Therefore, in average, generally stricly less than $(1-p^{10})\times 100$ woman in a hundred will have an unwanted pregnancy over a period of 10 years.

• Yes, this and the effect described by @user385948 both go in the same direction. – Mark L. Stone Mar 26 '17 at 17:20
• @MarkL.Stone I am having a little bit of difficulty reconciliating the problem posed by 'premature expectation' and the calculation of the probability of 'not getting pregnant over 10 years' for an individual, when the average success rate $p$ is my best guess for the individual's probability of not getting pregnant in a year. Can you point me towards an appropriate reference? – Olivier Mar 26 '17 at 18:44
• it's now in your addendum. Using the averaged probability in further calculations constitutes premature expectation. Jensen's inequality shows its consequence in this case. – Mark L. Stone Mar 26 '17 at 19:00
• To make more explicit, remember, every probability is an expected value ... of an indicator function. – Mark L. Stone Mar 26 '17 at 19:20
• Thank you for you answer, things are becoming clear. The problem is that $p = \tfrac{1}{M} \sum_w q_w$ is not a good guess for an individual's probability of not getting pregnant in a year. We must necessarily take into account the distribution of $q_w$. Thus $\tfrac{1}{M} \sum_w q_w^N$ is the marginal probability we are after, whereas $\left(\tfrac{1}{M}\sum_w q_w\right)^N$ is the poor man's plug-in estimate, which is biased. – Olivier Mar 26 '17 at 19:30

From a probabilist perspective I would expect that $$\mathbb P(\text{Not pregnant after year N}) \geq \mathbb P(\text{Not pregnant after year 1})^N.$$ This expectation is motivated as follows. Assume that at time $t=0$, every woman is assigned a (potentially different) number $p\in[0,1]$, the probability that she will get pregnant in the first year. If she did not get pregnant after $k$ years, then the probability that she gets pregnant in the $k+1$-th year is again $p$. Then $1-\mathbb{E}p$ is exactly $$\mathbb P(\text{Not pregnant after year 1}).$$ We want to prove that $$\mathbb P(\text{Not pregnant after year 1})^N$$ is a lower bound for $$\mathbb P(\text{Not pregnant after year N}).$$ But, given the number of women and $$1-\mathbb{E}p=\mathbb P(\text{Not pregnant after year 1}),$$ we can optimise the values for $p$ of the individual women to minimise $$\mathbb P(\text{Not pregnant after year N}).$$ There is one global minimum, it is "assign $p' =\mathbb{E} p$ to any woman" (so $p'$ is deterministic), and for this minimum we have equality (because indeed everything is independent). The inequality then follows. To illustrate this with an example, suppose that we have two women, having $p=0$ and $p=1$. Then $$\frac{1}{2}=\mathbb P(\text{Not pregnant after year N})>\mathbb P(\text{Not pregnant after year 1})^N =\frac{1}{2^N}$$ for $N>1$.

• Don't know who downvoted you - hopefully they will retract it.. Yes, this is Jensen's inequality applied to the convex function $p^N$ (as a function of $p$). This is what I was referring to in my comment above " Unwarranted independence assumption (often done in conjunction with premature expectation)". Premature expectation by averaging the probabilities over different population groups, then using independence assumption to get the final answer. There could be more wrong than just this though with the article methodology. – Mark L. Stone Mar 26 '17 at 17:07
• I downvoted because of this sentence: 'If she did not get pregnant after k years, then the probability that she gets pregnant in the k+1-th year is again p.' This reproduces the possibly erronerous year-to-year invariance of the probabilities, which I thought was the main problem with the NYT methodology. I was too fast on the downvote; I'll +1 when it is edited. This answer provides an interesting alternative point of view. – Olivier Mar 26 '17 at 17:22
• The idea is that the parameter $p$ is assigned to each woman separately, reflecting her or her partner's competence. This is assumed to be constant over time, which is inherently part of the model I set out. Note that in this model, the probabilities of someone not being pregnant up till year k and getting pregnant the year after are not independent because $p$ is not constant over all women. – user385948 Mar 26 '17 at 17:30
• Both effects can contribute, and both go in the same direction. It would have been better for both of your answers to have been combined into one. – Mark L. Stone Mar 26 '17 at 18:20