I have a basic stats question. So I am comparing a patient with a condition to the population of healthy. Due to only have 1 observation/patient, I am looking into the z-score, i.e. calculating that patient z-score based on healthy stats, to check how much it deviates to healthy central tendency and how significant it is (by converting it to p-value).

My questions:

  1. Can this approach be called a 'one-sample z-test'? I have been reading around and most of the example I found, the 'one-sample' refers to another sample population, not 1 observation.
  2. My follow-up question would be, is it valid to do multiple comparison correction, such as Bonferroni-Holm on top of this analysis?

I have been browsing about this but could not find any information for this specific question. Any help would be greatly appreciated!


It depends on whether you want to identify unhealthy by scores that are too large, or too small, or on either side, relative to healthy. Assuming "too large" indicates unhealthy, and assuming you have a large database of healthy, I would suggest using the empirical upper quantile (say 0.95) of the distribution of scores for healthy people. You can then make an "unhealthy" determination if the the score exceeds this quantile. I would not suggest using $t$ or $z$ test because the distribution of scores for healthy is not normal, and because with only one observation, there is no help coming from the Central Limit Theorem.

On the other hand, if you only have the summary statistics from the healthy group, you could use, e.g., $\bar x + 1.65 s$, where $\bar x$ and $s$ are the mean and sdev from the healthy group, as an estimate of the 0.95 quantile, but just realize that it might not be a very accurate estimate, depending on the sample size and on the degree of nonnormality.

This test does not have a "name" per se as far as I know, but a similar procedure is used in financial "event study" analysis.

As far as multiple comparisons goes, I would not suggest using them at all here. Instead simply acknowledge that there will be a 5% rate (or, say 1%, if you use the 0.99 quantile) of determinations of "unhealthy" among people who are actually healthy, and proceed with the appropriate caveats. Otherwise, you run the risk of missing too many unhealthy people, because the quantile thresholds automatically increase when you use multiple comparisons.

Even without multiple comparisons, you should also acknowledge that this procedure has a risk of not flagging an unhealthy person. If you have data on unhealthy people, you can easily estimate the probability of this outcome by using the proportion of people in the unhealthy group whose score is below the chosen quantile threshold.

Taking this further, you can determine an optimal threshold by using Bayesian decision theory.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.