# How to check hypothesis about estimation of random variable with unknown distribution?

I have one data sample of non-negative random variable $X$ with unknown distribution and predefined expected value $y$. Is there any test able to check null hypothesis $\mathbb{E}[X]\geq y$ or $\mathbb{E}[X]\leq y$?

Actual data samples are gathered in realtime. More specifically, it's an intervals between HTTP-requests coming to web-server from one client. Pearson's test shown that it is not normally distributed variable.

• How many values are in your sample and what level of significance do you need? – whuber Jun 4 '11 at 22:07
• Sample size varying from 5 to few hundreds. Level of significance - 0.05 – gelraen Jun 5 '11 at 6:36

The problem with allowing any distribution is that it could have a tiny chance of yielding a huge value. That eliminates any possibility of testing the mean with satisfactory confidence.

Here are the details. Choose a unit of measurement in which $y$ is hugely greater than $1$. Let $\alpha$ be the desired significance for the hypothesis test ($0 \lt \alpha \lt 1$) and $n$ be the sample size. Choose any $p$ for which $0 \lt p \lt 1 - \alpha^{1/n}$ and define $\mu = 1 + y/p$. Consider the two-point distribution for which $1$ has probability $1-p$ and $\mu$ has probability $p$. The chance that a sample of size $n$ from this distribution consists entirely of $1$s is

$$(1-p)^n \gt (\alpha^{1/n})^n = \alpha,$$

yet its expectation is

$$1(1-p) + \mu (p) = (1-p) + (1 + y/p)p = y+1 \gt y.$$

Because $y$ can be made arbitrarily large compared to $1$, no hypothesis test of any positive power will conclude that the true mean exceeds $y$ when $n$ $1$s are observed. Therefore the test will fail to detect that the mean exceeds $y$ with probability greater than $\alpha$ when this two-point distribution is the true distribution. Because this analysis places no restrictions on $\alpha$ or $n$, this proves that no test with positive power, with any amount of sampling, can achieve any positive level of significance.

Intervals between things like requests are often modeled well with exponential, gamma, and Weibull distributions. These can have pretty fat tails, so @whuber's concern is already accounted for, to some extent, when you calculate your confidence intervals.

Let's take an extreme example where you have just one value in your sample. Then you have no information about the dispersion of $X$ either from your knowledge of the distribution or from your sample and so no way of testing your hypotheses.

It does not take much to change this: for example if you know that $X$ is always non-negative then, taking the null hypothesis $\mathbb{E}[X] \le y$, by Markov's inequality you have

$$\Pr(X \geq x) \leq \frac{y}{x}$$

• Yes, I forgot to mention that $X$ is always non-negative. But still I don't see how to use Markov's inequality. Do I need to calculate cumulative distribution function from sample and check every value in sample to satisfy this inequality? – gelraen Jun 5 '11 at 6:58
• @gelraen: this was written before you said "Sample size varying from 5 to few hundreds" and assumed the sample size was 1. – Henry Jun 5 '11 at 8:54