# What statistical techniques to use to distinguish between two groups?

I have a dataset of about 300 people. 200 test positive for a disease, and the rest test negative. I have data on different test scores and imaging results for these 300 participants. So my dataset would look something like this

    status    test1   test2   test3   imaging1
pos        10      10      5        98
neg        8       7       5        77
pos        8       9       5        98
pos        10      10      5        99
pos        10      10      5        100
neg        6       8       4        78


And so forth. Is there a technique that tells me whether the two groups are different, and if so, how different? I know of t-test, but is there something else that can tell me that the positive group is different from the negative group (if so, what %)?

-
What do you exactly mean with "different" and "how different"? Do you want to be able to classify them and a confidence score? what's the task you want to solve? – jpmuc Jul 28 '14 at 7:16
@juampa Yes, how would I be able to do that? – Adrian Jul 28 '14 at 7:19
If the assumptions don't seem too badly violated (and they might be in this case), you might consider something along the lines of a two-sample T^2 statistic. – Glen_b Jul 28 '14 at 9:05

So you are looking for an algorithm which is able to test your a disease (classify) and deliver a confidence score. Furthermore, you'd like to see how confident the classification is.

A classifier to start with is Logistic regression. There are many packages that offer an implementation. Now, there are different ways to evaluate the resulting classifier. One that applies to your case is to estimate the confidence levels of the accuracy your classifier. See slide 22. Other possibilities are described here.

Just to be specific, the 95% confidence value for the accuracy (error rate) would be calculated as, $$E_{R}(h) \pm 1.96 \sqrt{\frac{E_{R}(h)(1-E_{R}(h))}{n}}$$ where $E_{R}(h)$ is the sample error rate, and $n$ the number of samples.

A common approach for clinical settings is evaluating the odds ratio. In Coursera there are a couple of statistics courses which address this question.

-

You are looking for a binary classifier. Have a look at the wikipedia article for an overview. To start with something simple I would suggest SVM. You would use your tests as your "features".

-