7
$\begingroup$

I come up from posts like What to do with a small (27) medical dataset? because they are fairly similar.

But in a broader context, in rare diseases/studies with a really low sample size -but plenty of features, wherein my example- there is no more than 30 instances for 4 classes (And they are really unbalanced 17-2-6-6, that can I even group into 17-8-6).

Is there any alternative to just look at the figures (scatterplots, hists, boxplots) of our features/variables to try to discern any potential biomarkers?

I have tried a PCA in order to see if they "cluster" in response to my class variable (disease/State of the disease). But appart from that and the plots for descriptive statistics Im really concerned that with this low sample size there is no statistical relevance we can assume (even a p-value with non parametric methods to compare "means"-because, in non parametric we are rather comparing the ranks- seems rather risky).

So, in this cases, where even a bootstrap-resampling doesn't do very much- where we have an extremely low sample and "all the features we want", is there any way to go?

PD: And if not: IS there a way to explain to the medics that collects the data -which is an important and hard work nonetheless- that I cant do anything remotely complete / worth to publish with this? Because sometimes i think they expect me to be a wizzard.

PD2: Btw the approach of the clinics ain't even to test a defined hypothesis, they are trying to see if any patterns/features can be used as biomarkers. Which makes my work harder.

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ I recommend you start with other questions tagged as small-sample. $\endgroup$
    – Tim ♦
    Sep 19, 2021 at 8:13
  • 1
    $\begingroup$ You can try to get inspiration from others in similar positions, e.g. this paper. $\endgroup$
    – Gumeo
    Sep 19, 2021 at 10:36
  • 2
    $\begingroup$ I think this situation is basically just going to be difficult. My guess is real progress is going to come from somewhere else: Can the condition be reproduced and studied in an animal model? Is anything known about the physiological processes involved? Does the condition appear similar to a better known disease? Etc. $\endgroup$
    – gung - Reinstate Monica
    Sep 20, 2021 at 3:22
  • 2
    $\begingroup$ Great question and unfortunately not an uncommon situation. One solution outside of data-analysis is to pool data with other institutions, a solution which has its own obstacles. $\endgroup$
    – Ivana
    Sep 20, 2021 at 9:40
  • 1
    $\begingroup$ You may want to look at our recent work with very rare diseases e.g. jidonline.org/article/S0022-202X(21)01506-2/fulltext It will give you some ideas about building a narrative using small amount of data. $\endgroup$
    – saint_667
    Sep 22, 2021 at 10:36

1 Answer 1

Reset to default
11
$\begingroup$

Such datasets are too small for reliable biomarker development. The only hope is a proof-of-concept study, which is really a proof-of-signal-existence study where you have to put all your eggs in one basket and examine the degree to which that basket relates to an outcome. The basket needs to be arrived at using unsupervised learning (aka data reduction), e.g., computing the first principal component of all candidate markers.

To put this into perspective, if you wanted to do the simplest thing possible, which is to estimate the probability of an outcome with no biomarkers and no other descriptor variables available, you have to be able to estimate a single probability. The minimum sample size needed to estimate a single probability with a margin of error of 0.10 (not very small, btw), requires n=96. So this is the minimum sample size for estimating a predictive model that only contains an intercept. If you had one candidate biomarker that was two-valued (binary biomarkers do not exist in nature) and you had the optimum situation where the number of negative biomarker values equaled the number of positive values, you'd need n = 96*2 = 192 subjects.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Maybe i have used biomarker a little bit broadly. As i biomarker i intended to explain like either a molecule (either mirna, or trigliricerid levels), or a sect of factors/conditions. So i will need, for that margion of error and just 1 biomarker/feature -or 1 PC-, 192 subjects. Thats not likely to happen in low-investment (public) research :C. Thank you for anwering Mr Harrel. Its very much appreciated $\endgroup$
    – Galpaccru
    Sep 19, 2021 at 15:42
  • 1
    $\begingroup$ Whatever you do, don't go beyond what the information content in the data supports. You may only be able to say how biomarkers relate to each other or how the first principal component relates to an outcome. $\endgroup$
    – Frank Harrell
    Sep 19, 2021 at 21:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.