Why is rejection of null hypothesis not a case of prosecutor's fallacy? Here is what my understanding is:
p-value - probability of finding the observed, or more extreme, results when the null hypothesis (H0) of a study question is true
which is to say p-value$=P(evidence/nullhypothesis)$. Now when the p value is below a certain threshold ($alpha$) we reject the null hypothesis.
I know i am missing something very basic here but, how is rejecting null hypothesis based upon low probability of evidence being what it is had the null hypothesis been true, not a case of committing prosecutor's fallacy?
 A: The p-value is the probability of seeing what you saw or something more extreme if the null hypothesis was true. The p-value is not the probability that the null hypothesis is true. So yes, interpreting a p-value as the probability that the null hypothesis is true is akin to the prosecutor's fallacy. If you want that probability, you need to assume a probability of the null hypothesis being true prior to the collection of data. Then you can use the data collected to influence or update that initial probability.
Whether or not choosing to "reject the null hypothesis" is akin to the prosecutor's fallacy gets into semantics. If "rejecting the null" means to believe the null is false in a probabilistic sense, then yes, that's commiting the prosecutor's fallacy. If "rejecting the null" means to act as if the null is false, that's different. That's a decision process whose performance will depend on the situations in which it's used.
A great example is the response of the scientific community to the first study showing evidence of a new particle with p < 0.0000003. Do all of the scientists accept the particle's existence? No. Some may, but some will remain skeptical. The differences in beliefs are connected to different prior probabilities on the null, i.e. how skeptical were they of the new particle's existence before the experiment. The results of one study can only shift their belief probabilities so far.
But what does the scientific community do? They do a second experiment. They act as if the particle exists, or more precisely, they act as though the existence of the particle warrants further study. Even the skeptical scientists will support acting in this way. If the second experiment also has a p < 0.0000003, some of the skeptical scientists will now believe the particle exists. Why? Even if the first experiment didn't convince them, it still shifted their belief probabilities. The second experiment will shift them further.
The second experiment may lead to a third, and so on. Each scientist's underlying belief distribution shifts with each experiment. After a given experiment, they may not agree on the existence of the particle, but still agree that the experiments are worth continuing. Eventually the series of experiments will have shifted all but the most skeptical scientists' belief distributions over to believing the particle exists.
Personal note: I'm not trying to sell anyone on this statistical paradigm; only to answer the initial question. There are other statistical paradigms worth exploring. Bayesian analysis facilitates explicitly quantifying your belief distribution before and after the experiment. Likelihood inference facilitates expressing the evidence of the experiment in a way that those with different prior beliefs can still agree on. Second generation p-values place the focus on pre-specifying clinical significance and providing clinicians with a value that behaves the way they wish the traditional p-value did, i.e. still indicating when the evidence is against the null but also distinguishing between when the evidence is for the null versus when uncertainty remains high. And there are many other interesting approaches.
