Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a dataset of about 300 people. 200 test positive for a disease, and the rest test negative. I have data on different test scores and imaging results for these 300 participants. So my dataset would look something like this

    status    test1   test2   test3   imaging1
    pos        10      10      5        98
    neg        8       7       5        77
    pos        8       9       5        98
    pos        10      10      5        99
    pos        10      10      5        100
    neg        6       8       4        78

And so forth. Is there a technique that tells me whether the two groups are different, and if so, how different? I know of t-test, but is there something else that can tell me that the positive group is different from the negative group (if so, what %)?

share|improve this question
What do you exactly mean with "different" and "how different"? Do you want to be able to classify them and a confidence score? what's the task you want to solve? – jpmuc Jul 28 '14 at 7:16
@juampa Yes, how would I be able to do that? – Adrian Jul 28 '14 at 7:19
If the assumptions don't seem too badly violated (and they might be in this case), you might consider something along the lines of a two-sample T^2 statistic. – Glen_b Jul 28 '14 at 9:05
up vote 3 down vote accepted

So you are looking for an algorithm which is able to test your a disease (classify) and deliver a confidence score. Furthermore, you'd like to see how confident the classification is.

A classifier to start with is Logistic regression. There are many packages that offer an implementation. Now, there are different ways to evaluate the resulting classifier. One that applies to your case is to estimate the confidence levels of the accuracy your classifier. See slide 22. Other possibilities are described here.

Just to be specific, the 95% confidence value for the accuracy (error rate) would be calculated as, $$ E_{R}(h) \pm 1.96 \sqrt{\frac{E_{R}(h)(1-E_{R}(h))}{n}} $$ where $E_{R}(h)$ is the sample error rate, and $n$ the number of samples.

A common approach for clinical settings is evaluating the odds ratio. In Coursera there are a couple of statistics courses which address this question.

share|improve this answer

You are looking for a binary classifier. Have a look at the wikipedia article for an overview. To start with something simple I would suggest SVM. You would use your tests as your "features".

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.