NOTE this answer precedes the OP's note about percentages. This answer assumed:
1/41 is the prior probability of infection in the tested population (not the population as a whole)
2/1000 is the false negative rate of the PCR test $P(PCR^{-}|Infected)$
I may be confused myself as my calculations do give probabilities summing to 1.
The analytical false positive rate of a PCR test is 2 in 1,000
I assumed you mean false negative rate, right? Otherwise how can you have 2.4% true positives (1/41), 6.8% positive tests but only 0.2% false positives?
Here's my reasoning. First let's set up a simple tree with 100k people (there is some rounding here):
100000 people
|
--------------------
| |
Infected Not infected
P = 1/41 P = 40/41
2439 97561
| |
------------- ------------
| | | |
PCR+ PCR- PCR+ PCR-
P = 0.998 P = 0.002 P = 0.044 P = 0.956
2434 5 4366 93195
Here P = 0.044 is the false positive rate and comes from 0.068 - (0.998 * 1/41).
And here's the probabilities:
$$ P(inf^{-}|PCR^{+}) = \frac{P(inf^{-}) \cdot P(PCR^{+}|inf^{-})}{P(inf^{-}) \cdot P(PCR^{+}|inf^{-}) + P(inf^{+}) \cdot P(PCR^{+}|inf^{+})} \\ = \frac{(40/41) \cdot 0.044}{(40/41) \cdot 0.044 + (1/41) \cdot 0.998} = 0.638 $$
and
$$ P(inf^{+}|PCR^{+}) = \frac{P(inf^{+}) \cdot P(PCR^{+}|inf^{+})}{P(inf^{-}) \cdot P(PCR^{+}|inf^{-}) + P(inf^{+}) \cdot P(PCR^{+}|inf^{+})} \\ = \frac{(1/41) \cdot 0.998}{(40/41) \cdot 0.044 + (1/41) \cdot 0.998} = 0.362 $$
If I'm correct, your mistake is in using 0.068 as denominator in the Bayes formulas.