# Hypothesis testing on an empirical distribution

This may be a basic question, so please pardon me: given a vector of univariate data, and an individual observation from the said univariate data, I would like to derive some probability of observing this individual observation, or a more extreme observation, from the empirical distribution of the said univariate data (aka. the p-value).

May I ask how can this be done in R?

First, let's start with $p$-value, because it seems you have a wrong understanding of $p$-values. They are not the "probability of observing this individual observation, or a more extreme observation, from the distribution". Quoting Wikipedia

$p$-value is a function of the observed sample results (a statistic) that is used for testing a statistical hypothesis. Before performing the test a threshold value is chosen, called the significance level of the test, traditionally 5% or 1% and denoted as $\alpha$. If the $p$-value is equal to or smaller than the significance level ($\alpha$), it suggests that the observed data are inconsistent with the assumption that the null hypothesis is true, and thus that hypothesis must be rejected and the alternative hypothesis is accepted as true. When the $p$-value is calculated correctly, such a test is guaranteed to control the Type I error rate to be no greater than $\alpha$.

An equivalent interpretation is that $p$-value is the probability of obtaining the observed sample results, or "more extreme" results, when the null hypothesis is actually true (where "more extreme" is dependent on the way the hypothesis is tested).

for learning more on $p$-values check this thread.

As about checking what is the probability of observing a certain value $x$ in your empirical distribution (i.e. data) - for this you simply count how many times $x$ occurred in your data.

data <- c(1,2,3,4,2,1,4,3,1,2,6,7,8,8,1,2)
x <- 5
mean(data == x) # P(data = x)
mean(data >= x) # P(data >= x)


however, let me say it again, this is not a $p$-value.

• Thanks for your answer Tim. I see now where my confusion with p-values lie now, thanks for your clarification, and the solution you provided to my problem! Commented May 15, 2015 at 7:41

It's possible to interpret your computation as a p-value if you want to 'test' whether the individual observation is from the same distribution as the available data. One could then consider the data as a bootstrap sample from the distribution. Your null hypothesis would be H_0: x ~ f_0, where x is the individual observation. Then your calculation could be looked at a bootstrap test, and your computation the p-value, because you are treating x as the 'test statistic' and you computation is being done under the null hypothesis.

On the other hand, the way you have written your question, you've just calculated the Prob. of a certain area, as the above answer points out.

• (1) To what "computation" are you referring? The question doesn't seem to describe one. (2) How would a "bootstrap" sample of a distribution differ from a sample of the distribution? (3) What is "f_0" in your null hypothesis? Remember, a null must refer to the underlying distribution, not to the data.
– whuber
Commented Feb 27, 2020 at 21:07