Feature selection for disease classification based on tests I have a dataset of around 100 different subjects
Some of them have a disease, some do not (roughly 60:40 disease:no disease)
They are subjected to a battery of 15 tests, to see if they are outside "normal" ranges.
Just plotting the values for the different tests for disease vs. non-disease as different colours (using matplot() in R), I can see that the different groups follow distinct patterns across the different features.
I then cluster the different groups (using hclust() in R) and if I cut the tree to make two clusters, the two groups separate fairly well into different clusters.
My aim is to devise a set of rules from these tests, so if we test a new patient, we can decide whether or not they have a disease.
So I need a classifier, to decide these rules, i.e. to work out which features to use, and what score cutoffs. What do people recommend?
 A: I just started taking the free probabilistic graphical models (PGM) course at  www.coursera.org ( Stanford). I highly recommend watching Daphne Koller's lecture on Medical diagnosis, which is part of her week one lectures. She specifically discusses David Heckerman's   (et. al) work on rule-based versus Bayes network approaches to medical diagnosis. In particular she discusses the ease of implementation and diagnosis accuracy improvement using a Bayes network when compared with their trying to use a rules-based approach. 
A: Like chl suggested, you need a supervised classifier. I'd suggest either linear SVM, or a decision tree such as C4.5. Both have libraries with a lot of documentation. Linear SVM would give you a "score-like" classifier, something like "2 points if test1 is positive", and "3 points if test2 is positive". The disease is present if overall score is above some threshold. A decision tree would give you a series of decisions, such as "if test1 is positive, do test2, otherwise do test4".
As for feature selection, there are lots of different methods, but not too many standard ones. If you use a tree, then it's likely that not all features will be used along any given path. So if you do the tests sequentially, you will need fewer than 15 for most patients. If you go with SVM, each test will get a score; you can dump features with smallest (in absolute value) scores and try to re-do the SVM with fewer features. Yet another feature selection method is by using mutual information, but that would depend on individual features having good mutual information with the class label.
