0
$\begingroup$

Hello I had a very interesting discussion and I need your help in clarifying the correct answer.

TLDR: Problem statement for a layman: Given I am a woman and I want to become professor, and given that $75\%$ of the professors are male - would it help if I fully control and change my gender by surgery, etc. to become a male because then the chances of becoming a professor would be higher (pretending the process was truly only influenced by the gender and there were no other influencing variables like publication success). Please forgive the placative example but this is the catchiest I could imagine to make this an interesting, more or less real life question.

More formal description: Given two discrete, binary random variables $X$ (gender, is male) and $Y$ (is professor), which have some joint probability $P(X,Y)$ which is unknown.

Let us have a conditional dataset with random samples where people $(x_i,y_i=1)$ with $i = 1,...,N,$ a big number.

Let's say I can estimate the conditonal probability almost perfectly (so I will not differentiate between estimated probabilities and true probabilities in the following).

This means from the dataset I infer $P(X\mid Y=1)$.

Now I also know Bayes theorem:

$$P(Y\mid X) =\frac{P(X\mid Y)P(Y)}{P(X)}$$

Now I want to know for an individual with full control over $X$ (i.e. not the population or a bigger sample) how to increase the individual probability $P(Y=1\mid X)$.

$$P(Y=1|X) =\frac{P(X\mid Y=1)P(Y=1)}{P(X)}$$

What to look at?

Update I should look at the relative risk https://en.m.wikipedia.org/wiki/Relative_risk which is not possible since P(X) is unknown in the dataset!

Ignoring the base rate P(X) while computing the relative risk is a form of base rate fallacy.

PS: A) I know that the dataset needs to be representative and may not suffer from selection bias. B) there may not be any confounders which would affect the outcome of $Y.$ C) if you ask how to interpret the probability of becoming a professor for an individual, the answer is parallel universes :)

$\endgroup$
2
  • 1
    $\begingroup$ I am not sure I understand your argument. Let's change the structure: suppose I had a ticket-based lottery and said that I have four equivalent prizes, but would award one prize by drawing from the green tickets sold and three by drawing from the red tickets sold, and everybody knows this. You can buy one ticket: would you buy a red or a green ticket? If you knew that so far $80\%$ of tickets sold have been red, would that affect your choice? $75\%$? $\endgroup$
    – Henry
    Sep 21 at 23:59
  • $\begingroup$ I am not sure your problem is equivalent. I feel the constraint to a fixed number of prizes and the involvement of other people's decisions makes it harder and non-equivalent $\endgroup$
    – Ggjj11
    Sep 22 at 0:07

1 Answer 1

1
$\begingroup$

I hope I understand the setup right but to me yes it does increase or decrease your chances of success on $Y$. I always like to think of Conditional Probabilities as simply just changing/shrinking the sample space. Consider the figure below of the binary events $X$ and $Y$ after fixing $X=1$, then surely your new sample space or the area you're thinking about will change and depending on the available information and you will be able to leverage the fixing of X to increase your chances. Although not the same problem but kinda reminds me of the Monte Hall problem.

Overall Shrunk

$\endgroup$
1
  • $\begingroup$ I was wrong. I should have read en.m.wikipedia.org/wiki/Relative_risk in general we are interested in P(Y=1|X=0)/P(Y=1|X=1)=[P(X=0|Y=1)P(Y=1)/P(X=0)]/[P(X=1|Y=1)P(Y=1)/P(X=1)] which involves the marginal frequency P(X). As soon as P(X) has a big imbalance, this starts to matter. $\endgroup$
    – Ggjj11
    Sep 30 at 18:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.