Is the p-value still uniformly distributed when the null hypothesis is composite? When the null hypothesis is simple (i.e., has only distribution of the sample) and the only distribution of the sample is continuous, the p-value can be shown to be uniformly distributed over (0,1).
When the null hypothesis is composite (i.e., has more than one sample distributions), is the p-value still uniformly distributed over (0,1)? Why?
 A: First of all let's deal with the simple point hypothesis. It is often claimed (as it is currently in Wikipedia) that the p-value is uniform over [0,1].
This is obviously not the case for discrete outcomes. For example if a coin is tossed 5 times and the test statistic is the number of heads obtained there are only 3 possible p-values under the null hypothesis that the coin is fair. The p-values 0.0625, 0.375 and 1 have probability 0.0625, 0.3125 and 0.625 of occuring arising from outcomes {0,5}, {1,4} and {2,3} respectively.
Perhaps less obvious is that it is not always true for the continuous case. Consider the the null hypothesis that the location of the minute hand on my clock when the battery runs out is uniformly distributed over the range [0,60). All outcomes have the same probability of occuring so the probability of getting an outcome as extreme or more extreme than the observed outcome is unity.
Moving on to composite nulls. No the p-value is not distributed over (0,1) for composite nulls and not just for the reasons given above. Perhaps the p-value has no meaning at all for composite nulls.
Alternatively the p-value is taken to be the p-value for the most favourable member of the composite null for the observed data i.e. the maximum likelihood member of the null.  Under this interpretation a p-value of zero may not be possible. Consider the null hypothesis that a coin is fair where fair is defined with a bit of tolerance so that 0.49 < p < 0.51. If zero heads are observed the most likely point hypothesis is p=0.49. But using this member of the null there are more extreme outcomes: in this case all heads would be even less probable. Hence there is a lower bound on the p-value which is not 0.
Note that considering the least favourable member of the null for a given dataset doesn't get you anywhere - for some composite hypotheses it will always be possible to choose a member of the null for which the probability of the observed data is zero whatever the data may be (and therefore a p-value which is always zero). An example is the null that the coin is biased where biased is defined as outside of the above range i.e. p<0.49 or p>0.51. If some heads are observed the point hypothesis p=0 renders this impossible. Conversely if no heads are obtained p=1 renders this impossible. Therefore p-value cannot be defined in terms of least favourable member of the null.
A: The answer seems to be 'No' but sometimes 'Yes, asymptotically, depending on the construction of p'. Robins, van der Vaart, and Ventura (2000) go through the details.
The reasoning is basically what @whuber states telegraphically in the comment.  The state of nature matters to the nuisance parameters.
