You wrongly assume
$$\begin{array}{}P(\text{positive test given infected}) & =& 1 - P(\text{positive test given not infected}) \\ &=& 1-0.002 \\&=& 0.998 \end{array}$$
You are only given the information $P(\text{positive test given not infected}) = 0.002$ but not $P(\text{positive test given infected}) = 0.998$.
So your first half of the reasoning seems correct.
$$\begin{array}{}P(\text{not infected | positive}) &=& \frac{P(\text{positive | not infected})\cdot P(\text{not infected})}{P(\text{positive})} \\ &=& \frac{0.002 \cdot 0.976}{0.068} \\ &=& 0.0287 \end{array}$$
However we get strange values when we try to deduce $P(\text{positive | infected})$ from
$$\begin{array}{rcl}P(\text{infected | positive}) &=& \frac{P(\text{positive | infected})\cdot P(\text{ infected})}{P(\text{positive})} \\ 0.9713&=& \frac{? \cdot 0.024}{0.068} \end{array}$$
or alternatively from
$$\begin{array}{} P(\text{pos}) &=& P(\text{pos | $\neg$ inf})\cdot P(\text{$\neg$ inf}) + P(\text{pos | inf})\cdot P(\text{ inf})\\ 0.068 &=& 0.002 \frac{40}{41} + P(\text{pos | inf}) \frac{1}{41} \end{array}$$
or
$$P(\text{pos | inf}) = 0.068 \cdot 41 - 0.002\cdot 40 = 2.708$$
Which is not possible. Intuitively we can see this more directly... If only 1 in 41 people are sick (2.43 %)then it is strange to find that 6.8% of the tests are positive. The percentage of positive tests is much higher than the people that are sick.
This can be explained in two ways:
- The false positive rate is not correct (it should be much higher and somewhere close to this 6.8%).
- Among the people that are performing tests there are a higher proportion of sick people (higher than 1 in 41).