In most examples I find, the various probabilities are just there and it is a simple question of applying Bayes rule to those numbers. However, I'm having a hard time finding details about how to derive/find/discover those probabilities in my use case.
In my case, I have a dataset of support cases - someone who needed help. I have access to the full "population" if you will - all the cases. Once a case is closed, a customer can provide feedback(survey) indicating if they were satisfied or not. Not satisfied would in this case be the "cancer". I want to use Bayes rule to determine the probability that a case will/would get negative feedback or "have cancer". Not every case gets a survey returned, but I have all the surveys and all the cases connected to them.
In my mind, a returned survey is the thing that can be used to definitively label the connected case as negative (cancer) or positive (no cancer), so can be used to help determine the probabilities to use in Bayes rule.
Subsequently, I can use information I have about the case for my "test". For example, did the case take longer than x days to resolve? If it went over x days, that would be a positive test and under x days a negative test result.
So, given I have a dataset of what has happened in the past, how do I create the probabilities to use and apply Bayes rule?
What I've Got
Total Cases: 18942
Total Surveys: 1421 (1297 Positive, 124 negative)
Of 124 negative surveys, 39 were over x days
Of 1297 positive surveys, 111 were over x days
Here is what I've got with my limited understanding:
Prior probability of negative survey (cancer): [Negative Surveys] / [Total Cases] = .0065466
For my test:
Sensitivity (true positive): [Negatives that were > X days] / [All Negatives] = .3145161 False Positive : [Positive surveys > X days] / [All Positives] = .0855821
Now putting that into Bayes Rule, I have:
$P(A) = .0065466$
$P(B | A) = .3145161$
$P(B | ¬A) = .0855821$
Resulting in: (I think...)
$P(A | B) = .023644982$