2
$\begingroup$

I have an odd question concerning the way one could assess that a value collected on a single individual differs from values collected on a group of people N (clinical research). This is a lot like assessing that an individual is considered as an outlier I guess.

I was just thinking of two ways of doing it:

  • firstly one could just calculate the mean and standard deviation of the group (N values) and then assess the number of standard deviation the score of the individual is located from the group;

  • secondly I was just thinking of performing a one sample t test where the mean of values collected on the group differ from the single value collected on the individual (instead of testing against 0, testing against the value of the individual).

I would like to know if this would be a correct approach or if there might be other tests I do not know about.

Thank you!

$\endgroup$
  • $\begingroup$ To be clear about what you are asking, do you want to (a) test an individual who was singled out for such testing for some a priori reason unrelated to the data; (b) test all individuals to identify one who is "most outlying"; (c) test all individuals to identify any that appear to be outliers; or (d) something else? $\endgroup$ – whuber May 30 '17 at 18:14
  • 1
    $\begingroup$ For a priori reasons unrelated to the data (pathological case). $\endgroup$ – user5084 May 30 '17 at 18:33
2
$\begingroup$

You're basically interested in knowing how probable your test value $V$ is given the distribution that generated the $N$ other values. Also, from your selected tests, it sounds like we are testing for extremeness of a value relative to others (as opposed to, for example, inliers, or "splitting two modes").

I'd suggest against using either approach you gave. The former only makes sense if your population is well-approximated by a normal. However, the distribution of the $N$ values can really be almost anything (Cauchy, Exponential, etc), so the z-score may be meaningless. (Although Chebyshev Inequality gives rough bounds if you can estimate the mean and standard deviation well enough).

The t-test is not useful here because you cannot (or maybe don't want to) assume the null distribution of either the $N$ values nor your single value $V$ come from an approximately normal distribution.

Why not just compute the percentile of your test value $V$ relative to the $N$ values?

If we assume that your $N+1$ values are sampled iid from the same underlying distribution $P$ with CDF $F$, then $P(F(V)>0.99) < 0.01$. We can use the $N$ values to develop the empirical distribution function for $F$, from which we get the estimate of the percentile of $V$ as $\hat{F}(V)$.

If $\hat{F}(V)>0.99$ we can reject with approximately $99%$ confidence.

Update based on comments

As pointed out by @whuber, a parametric approach will probably be a better use of your data (provided you can justify some parametric family). If your data are reasonably normal, then you can fit a $t$-distribution to your $N$ points. The cool thing about the $t$-distribution is that it is the exact predictive distribution for a new data point, given the observed data come from a normal distribution.

So, let's say you have a sample of $N$ points from some Normal distribution, but you don't know the mean $\mu$ or standard deviation ($\sigma$). However, you have estimates of the mean and standard deviation using your $N$ points (i.e., $\hat{\mu} = \bar{x},\;\hat{\sigma}=s$), and you want to use these estimates to construct an interval that will contain the next value with some probability (say, 95% chance).

You can use the $t$-distribution to construct your (central) prediction intervals for the new observation $y$ from the N existing data points $\mathbf{x}$like so (central means its symmetric about the sample mean):

$$ \mathrm{Let}\;\;I_{\alpha,N} := \bar{x} \pm t_{\alpha/2}s_N\sqrt{1+\frac{1}{N}},\; \mathrm{then}$$

$$P(y \in I_{\alpha}) = 1-\alpha$$

So, in your case, you have your $N$ points and your $y$ is the "outlier" to be tested. You can set $\alpha$ to, say, .05 or .01. Then, if your point is outside this interval, it means that a point at least this extreme only happens 0.05 or .01 of the time, and so seems "unusual" give the other data.

That is one way forward.

$\endgroup$
  • 2
    $\begingroup$ There are many good thoughts here, +1. But is it really necessary even to refer to the empirical distribution? After all, $\hat{F}(V) \gt 0.99$ is equivalent to $V$ being in the top $1\%$ of the data--that's all; and the computation is simple. With that insight, unless $N\gg 100$, this test doesn't seem to help much, so maybe some distributional assumptions might be worth considering in that case. That leads one to suspect that perhaps the best course of action would simply be to plot the data and examine where this particular individual $V$ is situated in the plot. $\endgroup$ – whuber May 30 '17 at 21:04
  • 2
    $\begingroup$ @whuber You're right... one needs quite a bit of data to make the non-parametric approach really useful. The percentile is really the important quantity, and the ECDF is simply a representation of the percentiles (maybe it was explanatory overkill on my part...). One would hope that the OP has some sense of the overall shape and extremeness of the distribution. If not, then basic EDA or other subjective approaches are needed (as you suggest). $\endgroup$ – user145807 May 30 '17 at 21:33
  • 1
    $\begingroup$ It depends on how you apply the t-test. Done correctly, it will be equivalent to comparing the single value to a normal-theory prediction limit constructed from the remaining values. If you treat the single value as a constant, though (as suggested by the wording in the question), the test will give erroneous results. $\endgroup$ – whuber Jun 1 '17 at 14:12
  • 1
    $\begingroup$ @clowny as suggested by whuber, you're trying to assess if your observed value would be reasonable assuming it came from the distribution of the other data. A t-test will estimate a predictive distribution, assuming the data came from a normal distribution. So, just test if your new value is probable given the predictive distribution (t-dist) and you will have some basis for an objective assessment of "extremeness" $\endgroup$ – user145807 Jun 1 '17 at 15:40
  • 1
    $\begingroup$ @clowny sorry, I think my response was hopelessly jargon-y. I meant the opposite -- if your data appear reasonably normal, then a t-test should do the trick. I'll update my post. $\endgroup$ – user145807 Jun 2 '17 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy