In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is short for stochastically dominating a standard uniform random vairable), super-uniform and, exclusively used in the context of p-values (I think), the term of conservative p-values. However, I am no expert in this and am hoping someone can confirm and maybe corroborate with more or better or original sources.
Also note: I have tried to keep notation as it is in the original sources, so that either mistaken assumptions of equivalence that I made stand out starkly for the expert, or notation can be identified as equivalent wherever correct -- please feel free to comment on this, as well.
stochastically larger than uniform
Quirk and Saposnik, 1962 / Wikipedia on "Stochastic dominance"
The Wikipedia article on "Stochastic dominance" defines the "canonical first-order stochastic dominance (FSD)" as:
Random variable $A$ has first-order stochastic dominance over random variable $B$ if for any outcome $x$, $A$ gives at least as high a probability of receiving at least $x$ as does $B$, and for some $x$, $A$ gives a higher probability of receiving at least $x$. In notation form, $P[ A ≥ x ] ≥ P[ B ≥ x ]$ for all $x$, and for some $x$, $P[ A ≥ x ] > P[ B ≥ x ]$.
In addition, this cites an article for this defition:
Quirk, James P., and Rubin Saposnik. 1962. “Admissibility and Measurable Utility Functions.” The Review of Economic Studies 29 (2): 140–46. https://doi.org/10.2307/2295819.
In Definition 2 it gives the The Stochastic Dominance Ordering as:
2a. Let $\Lambda$ denote a set of probability distributions of income. A weak partial ordering $D$ is defined over $\Lambda$ as follows:
(i) For all $g', g^{"} \in \Lambda, g' D g^{"}$ (read as "$g^{"}$ is dominated in sense No.2 by $g'$") if and only if the corresponding cumulative distribution functions, $G'$ and $G^{"}$, satisfy:
$G' (y) ≤ G^{"}(y)$ for all $y$.
(ii) For all $g', g^{"} \in \Lambda, g' O_d g^{"}$ (read as "$g^{"}$ is not comparable in sense No.2 with $g'$") if and only if neither $g' D g^{"}$ nor $g^{"} D g'$ holds.
2b. For any $g^{"} \in \Lambda$, if there exists $g' \in \Lambda$ such that $g' D g^{"}$ and not $g^{"} D g'$, then $g^{"}$ is said to be stochastically dominated by g'.
The cumulative distribution function inequality gives a nice intuition for stochastic dominance, i.e. that the dominating distribution has less mass / area under the curve to the left of any point $y$ or that the integral up to any point is always smaller than that of the distribution it dominates.
In addition, an interesting footnote on the last cited paragraph above establishes that stochastical dominance is the same as stochastically larger:
Stochastic dominance, as defined here, is the same as "stochastically larger", as used in the Mann-Whitney test. See Fraser (2) pp. 160 ff.
University of Warwick, APTS teaching materials
The "Statistical Inference" course materials from the Academy for PhD Training in Statistics (APTS) at Warwick University (that current version by Simon Shaw, but I have also found an earlier version by Jonathan Rougier with similar content), nicely lays out what they mean by stochastically dominates, including a visual intuition for it:
Let $X$ and $Y$ be two scalar random variables. Then $X$ stochastically dominates $Y$ exactly when
$\mathbb{P}(X ≤ v) ≤ \mathbb{P}(Y ≤ v)$
for all $v \in \mathbb{R}$. Visually, the distribution function for $X$ is never to the left of the distribution function for $Y$.
super-uniform
For super-uniform I have found the following definitions, the first one explicitly working with the concept of stochastic dominance:
University of Warwick, APTS teaching materials
The APTS Warwick University material (same as cited above) establishes stochastical dominance over the standard uniform distribution by first recalling a basic property of that distribution over $[0,1]$:
Recall that if $U ∼ \text{Unif}(0, 1)$, the standard uniform distribution, then $\mathbb{P}(U ≤ u) = u$ for $u \in [0, 1]$.
Then they define super-uniform in Definition 21 (Super-uniform) with equation (4.2):
The random variable $X$ is super-uniform exactly when it stochastically dominates a standard uniform random variable. That is:
$P(X ≤ u) ≤ u$
for all $u ∈ [0, 1]$.
As stochistically larger is the same as stochistically dominates (see above), the stochistically dominates a standard uniform random variable here should mean the same as the commonly used stochastically larger than uniform, right?
Chen 2018
Just to chuck in another (somewhat independent?) place where super-uniform is defined, here is a very recent preprint that defines it:
Chen, Xiongzhi. 2018. “False Discovery Rate Control for Multiple Testing Based on P-Values with Cadlag Distribution Functions.” ArXiv:1803.06040 [Stat], March. http://arxiv.org/abs/1803.06040.
Its Definition 1 reads (basically identical to the Warwick definition above):
A random variable $X$ with range $[0, 1]$ is called “super-uniform” if
$Pr(X ≤ t) ≤ t$
for all $t ∈ [0, 1]$,
[...].
conservative p-values
I have seen p-values referred to as conservative p-values as in the distribution of these p-values increasing towards $1$. See e.g. Scenario D in How to interpret a p-value histogram on varianceexplained.
Is this colloquial usage of conservative p-values valid and if so, is it to commonly used?
And does it mean something like the the conservative p-values mentioned in this paper:
Zhao, Qingyuan, Dylan S. Small, and Weijie Su. 2018. “Multiple Testing When Many P-Values Are Uniformly Conservative, with Application to Testing Qualitative Interaction in Educational Interventions.” Journal of the American Statistical Association, July, 1–14. https://doi.org/10.1080/01621459.2018.1497499.
They state that:
However, in the motivating examples above as well as many other applications, it is common that the majority of the null p-values may be very conservative (stochastically larger than the uniform distribution).
Thus, they equate conservative p-values with stochastically larger than the uniform distribution, which is the concept I looked at above and would mean it is the same thing, right?
Edit / added later:
stochastically lower bounded by a uniform
I have just come across another formulation in the context of multiple testing. In full, it reads: stochastically lower bounded by a uniform random variable on [0, 1]
This is right below Definition 2.1 in:
Blanchard, Gilles, and Etienne Roquain. 2008. “Two Simple Sufficient Conditions for FDR Control.” Electronic Journal of Statistics 2: 963–92. https://doi.org/10.1214/08-EJS180.
