How to examine predictive value of variables for an outcome that can only be observed as a group?

Question

Suppose I have a bunch of people who did a few tests (let's call it tests A). Now I'll do a second test (test B) in order to categorize these people into two categories (that have a pre-defined "true" answer).

Note that score from test A can only predict the signal-to-noise ratio of test B. I.e you can't use score from test A to do the categorization itself.

My aim is to select a group of people that are as easily categorized into correct categories as possible (I.e actual score of test B does not matter), using the score of test A.

Now, I want to examine how test A (which consists of multiple variables A1, A2...) predicts the final outcome.

How could I do that? I thought about creating bunch of subgroups of a fixed sample size randomly and measure the mean of test score A and the percentage I got the categorization right for these subgroups, and then apply regression analysis on that. Would this be a valid approach? If not, what is the correct way of doing this?

Answer 1

This sounds like a trivial classification problem. The variables A1,A2,... will be your data and the true categories are dependent to these data. You can start with trying off-the-shelf classifiers, e.g. http://www.csie.ntu.edu.tw/~cjlin/libsvm/ or http://www.mathworks.com/help/toolbox/stats/classify.html.