2
$\begingroup$

A recent NYTimes article states that

The minuscule chances of the two highest-ranking F.B.I. officials — who made some of the most politically consequential law enforcement decisions in a generation — being randomly subjected to a detailed scrub of their tax returns a few years after leaving their posts presents extraordinary questions.

Was it sheer coincidence that two close associates would randomly come under the scrutiny of the same audit program within two years of each other?

According to the article, the probability that a given person is selected for a detailed audit in a given year is $p \approx \frac{1}{19250}$. (In reality this probability might vary slightly by year, but for the sake of simplicity let's assume it equals $p$ in every year.)

The article states that the two individuals in question were audited in different years, but within the same 4 year period of interest (coinciding with a presidential term).

What is the probability of the event "both of these people of interest are selected for an audit within a specified 4 year period"?

My first pass at an answer: the probability that a given person is not selected for an audit within a given 4 year period is $(1-p)^4$, assuming independence over time, and the probability of being selected for an audit once or more within the 4 year period is therefore $1 - (1-p)^4$. Assuming independence between individuals, the probability that both individuals are audited once or more within the 4 year period is therefore

$$ (1 - (1 - p)^4)^2 \approx \frac{43}{10^9} $$

or approximately 43 out of one billion.

Is that answer (approximately) correct?

The article goes on to ask whether "something in their returns increase[d] the chances of their being selected" for the audit. If the IRS's audit selection process isn't a simple random sample of individuals/tax returns, but instead varies with observable variables like income (perhaps high-income individuals are more likely to be selected for an audit), could that meaningfully change the answer (for example, by changing the answer to something larger than $0.001$)?

Edit: anyone reading this will likely be interested in this companion article.

$\endgroup$
5
  • 1
    $\begingroup$ Separately from the "what if $p$ varies with income" line of thinking, perhaps another way to extend the problem is to say "there were $K>2$ individuals whose selection for an audit might have appeared suspicious -- what is the probability that at least 2 out of $K$ of those people were selected for an audit at some point in the 4 year period?" $\endgroup$
    – Adrian
    Jul 7, 2022 at 0:54
  • 1
    $\begingroup$ This article is extremely misleading. People do get struck by lightning multiple times and people do win large lotteries within a short space of time. Formulating such probability statements post hoc leads to hugely wrong probability estimates. Read David Hand's book, The Improbability Principle, for an extensive account of eight reasons why such reporting is so problematic. Note, in particular, that answers to your specific probability questions are sensitive to unsupported assumptions such as the period of time to cover. Why shouldn't that period be 100 years instead of 4, e.g.? $\endgroup$
    – whuber
    Jul 7, 2022 at 14:14
  • 1
    $\begingroup$ If we further ask a more relevant question, such as "what are the chances that eventually some national newspaper reporter might find two people in the public eye who were randomly audited by the IRS in this way within a period the reporter finds interesting" (which accurately describes the present situation), we would realize (a) the question is vague and (b) when made more precise by specifying what "in the public eye" and "interesting" mean (along with making suppositions about the capabilities of national reporters), the answer is likely close to 100%. $\endgroup$
    – whuber
    Jul 7, 2022 at 15:38
  • 1
    $\begingroup$ See the companion article at nytimes.com/2022/07/07/upshot/comey-mccabe-tax-audits.html for a much better statistical assessment. 'Mr. Gelman, like every other statistician who spoke with The Times about this problem, said the biggest hurdle was not any of the details but defining the question itself. ... formulating the question is tricky, bordering on “meaningless,”. ... In this case, the best question is not one with an answer you can look up in a statistics textbook. Instead, ... the question to pose is: “What’s going on?” ' $\endgroup$
    – whuber
    Jul 7, 2022 at 21:31
  • 1
    $\begingroup$ I agree, the companion article is better. "It’s incorrect to narrow our search only to Mr. Comey and Mr. McCabe, because it’s likely we’d be examining these probabilities if we learned that two other notable political enemies of an administration were audited instead of these two men. A better question is: What is the likelihood that two or more people like Mr. Comey and Mr. McCabe would be audited over this period?" That agrees with your comment about "two people in the public eye", and my comment about "$K > 2$ individuals whose selection for an audit might have appeared suspicious." $\endgroup$
    – Adrian
    Jul 7, 2022 at 23:21

1 Answer 1

1
$\begingroup$

Your calculation of the probability $q$ of them being selected in the four-year period is correct.

And you are also right that this could be increased if the probability of being selected would be increased from $p = \frac{1}{19250}$ to something higher according to reasons like being suspicious or high income.

How high would $p$ have to be to get a $q>0.001$? $$ \begin{align} q &= (1 - (1-p)^4)^2\\ \leftrightarrow \sqrt q &= 1 - (1-p)^4\\ \leftrightarrow 1-\sqrt q &= (1-p)^4\\ \leftrightarrow \left(1-\sqrt q\right)^{1/4} &= 1-p\\ \leftrightarrow p &= 1 - \left(1-\sqrt q\right)^{1/4}. \end{align} $$ Thus, for $q = 0.001$, we get: $$ \begin{align} p &= 1 - \left(1 - \sqrt{0.001} \right)^{1/4}\\ &\approx 0.008. \end{align} $$ I.e. if the selection probability of an individual for audit in a given year would increase above 0.008, then the described event could happen with a probability higher than 0.001.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.