7
$\begingroup$

I have a set of samples in which I assume there are 2 definite subsets in it. I plotted their values in a histogram and found that there are two distinct modes as shown in the figure below.

My question is how do I differentiate two groups. i.e how do I choose a value that differentiates the two subsets?

enter image description here

$\endgroup$
5
  • 3
    $\begingroup$ This is a mixture modeling problem. Naming it is half the battle - I'll let somebody else take care of the other half. $\endgroup$ Commented Apr 26, 2011 at 23:37
  • $\begingroup$ If you're comfortable with R, check out the CRAN Task View on clustering - just searching the page for "mixture" will help you find related packages. $\endgroup$ Commented Apr 26, 2011 at 23:40
  • $\begingroup$ I don't want to use unsupervised clustering here. What about using Likelihood-ratio test? Is it a possible solution? if so, is there anyway to do that in R? $\endgroup$
    – Jana
    Commented Apr 27, 2011 at 0:00
  • $\begingroup$ If this is your only variable, you're going to do some sort of unsupervised learning, and since you're trying to divide it into two groups, you're going to do some sort of clustering. The relevant techniques that aren't called "unsupervised clustering" were probably just invented by statisticians instead of machine-learning people. $\endgroup$ Commented Apr 27, 2011 at 10:59
  • 1
    $\begingroup$ you tagged your question "statistical-significance". Does your question have such an aspect as well? $\endgroup$
    – GaBorgulya
    Commented Apr 27, 2011 at 11:44

2 Answers 2

3
$\begingroup$

I assume you are talking about Neonatal Behavioral Assessment Scale values in Hereditary Renal Adysplasia.

I often see in medical research that physicians want to have cut-offs and simple threshold based interpretations of their research results, based merely on the distribution of the measurements. Practice and applications however usually need high positive predictive value or high negative predictive value, so the characteristics of the future population tested have to be considered. My point of view is even if now you just want to "differentiate two groups" you probably want to apply this somehow in the future and thus you probably want to find the optimal threshold, optimising costs, risks and benefits (survival, quality of life etc.) in a practical setting. So I suggest that you to think these over in your application.

$\endgroup$
1
  • $\begingroup$ +1 for adding context and the interesting points you raised. Is this a validated scale? If so, I assume a cutoff has already been established because it should not be sample-dependent, for interpretation purpose. Good point about the PPV and prevalence. $\endgroup$
    – chl
    Commented Apr 27, 2011 at 20:05
0
$\begingroup$

If you are willing to assume the populations have the same variance you could use essentially LDA without the normality assumption (a.k.a. Fisher's Method or Fisher's Discriminant Function).

Without this assumption you could try an EM algorithm which is indirectly what Matt Suggested since this would be a mixture model approach.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.