2
$\begingroup$

I recently came upon a discrepancy between Matlab's classify and ClassificationDiscriminant functions. By default, they should both be doing Linear DiscriminantAnalysis, but when given unbalanced classes, they give very different results.

I have created a small test script here at Gist which creates some data that is unbalanced but easily separable. I test my classifiers on the training data for optimal results. For a linear classifier, the result should be unique, but it appears this is not the case.

$\endgroup$
3
$\begingroup$

The two functions do slightly different things.

According to the docs, classify uses a uniform prior (i.e., each class is equally likely). On the other hand, it looks ClassificationDiscriminant uses an empirical prior (i.e., the proportions of each class in your training data).

To extend your example on Gist:

N1 = 3500; N2 = 1500;
data = [randn(N1, 1) ; 1 + randn(N2, 1)];
labels = [zeros(N1, 1) ; ones(N2, 1)];

y_pred = classify(data, data, labels);
TP1 = sum(y_pred == 1 & labels == 1); %TP1 is now 1033

temp = ClassificationDiscriminant.fit(data, labels);
y_pred2 = temp.predict(data);
TP2 = sum(y_pred2 == 1 & labels == 1); %TP2 is 582

If you want them to match, you have two options. You could force ClassificationDiscriminant to use a uniform prior, like so:

temp.Prior = [0.5 0.5]
y_pred2 = temp.predict(data);
TP2 = sum(y_pred2 == 1 & labels == 1); %TP2 is now 1033, matching TP1

Alternately, you could provide the empirical prior to the classify function, like this:

y_pred = classify(data, data, labels, 'linear', [N1 N2]./(N1+N2));
TP1 = sum(y_pred == 1 & labels == 1); %TP1 is now 582

You could also pass the string 'empirical' as the last argument to classify for the same effect. Here are links to the relevant documentation:

$\endgroup$
  • $\begingroup$ Thanks a lot! I was looking through the docs, but must have missed this vital detail. $\endgroup$ – dvreed77 Feb 18 '13 at 23:59
  • $\begingroup$ Glad I could help! It's a subtle gotcha and I sometimes wish The Mathworks were a little more consistent about everything. $\endgroup$ – Matt Krause Feb 19 '13 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.