While reading a textbook on probability I came across a question:
There is a probability that a certain percentage of the population has a disease. Then a test for this disease could have a certain specificity (let's say .98) and sensitivity (let's say .99).
How do we know what percentage of population will have a disease, unless we test it? What other information/knowledge is used to estimate the percentage of population which will contract a disease?
You are correct to suspect that prevalence of a disease is sometimes estimated from test results.
Denote prevalence $\pi = P(D),$ sensitivity $\eta = P(+|D),$ specificity $\theta = P(-|D^c),$ and the probability of a positive test in the population as $\tau = P(+).$ Then $$\tau = \pi\eta + (1-\pi)(1-\theta).$$ Upon solving for $\pi,$ this implies $$\pi = \frac{\tau+\theta -1}{\eta+\theta - 1}.$$ So if you get the proportion $t = a/n$ of positive tests among $n$ randomly selected members of the population, you can estimate $\tau$ by $t$ and $\pi$ by $$p = \frac{t+\theta -1}{\eta+\theta - 1}.$$
If you want a confidence interval for $\pi,$ begin by getting the usual binomial confidence interval for $\tau$ and then use the second displayed equation on the endpoints of the CI for $\tau$ to get endpoints of a CI for $\pi.$
Note: Unfortunately, for tests with poor sensitivity or specificity or for prevalence $\pi$ near to $0$ or $1,$ the CI for $\pi$ can have nonsensical endpoints outside of $(0,1).$ Then a Gibbs sampler may provide a useful Bayesian probability interval for the prevalence $\pi$ of the disease.
You never actually know the truth.
Looked at one way, the logic of test accuracy, the 2×2 table, sensitivity, specificity, positive predictive value and all that is predicated upon a testing method under study compared to a "gold standard". For example, an oncology test based on lab work for a urinary biomarker might be compared to a specific protocol for biopsy and histology of related tissue. Of course, the protocol for biopsy and histology itself has some sensitivity and some specificity! In practice, though we might take such a protocol as ground truth, and rely on it as a gold standard... at least until something we feel is better can come along.
Looked at another way, the logic of test accuracy, the 2×2 table, sensitivity, specificity, positive predictive value and all that, irrespective of the quality or existence of any "gold standard", helps us organize our understanding of trade-offs, and the behavior of tests. For example, regardless of how close we can actually come to knowing The Truth of whatever we are trying to test, we can know that making the definition of a positive test easier necessarily increases our false positives (i.e. of $\alpha$ error/Type I error). Likewise, this logic can help us understand that positive predictive value of a test tanks when the prevalence of the thing we are testing for is tiny.
