4
$\begingroup$

I am evaluating a Quadratic Discriminant Analysis (QDA) classifier on a high-dimensionality feature set. The features come from highly non-Gaussian distributions. However, when I transform the features to each have a Gaussian distribution in two different ways, the resulting classifiers performs worse than QDA applied on the raw features with the following three metrics:

  • Accuracy (classes are balanced)
  • Area under the ROC
  • A probabilistic metric

The first way of transforming each feature to a Gaussian distribution disregarded the class. For each feature, it found the parameters of the the corresponding distribution, used the distribution's CDF function or approximation for each data point, and then used the Gaussian's inverse CDF.

The second way did the same thing, but included class labels and treated data from each class independently.

Any idea why this occurs?

I have confirmed that it is not due to ...

  1. A bug in the code
  2. Distributions changing from train set to test set
$\endgroup$
6
  • $\begingroup$ Is the performance measured with the training dataset or with a held-out dataset? $\endgroup$
    – whuber
    Mar 13, 2011 at 20:50
  • $\begingroup$ Measured both; trend holds true for both. $\endgroup$
    – benhamner
    Mar 13, 2011 at 21:06
  • 1
    $\begingroup$ If you only do marginal calibration, It might be that before tranforming the data quadratic separation holds and after transforming the data it does not hold anymore. $\endgroup$ Mar 13, 2011 at 21:45
  • $\begingroup$ I did not get how you have dealt with the covariance of the features. $\endgroup$
    – Wok
    Mar 13, 2011 at 22:41
  • $\begingroup$ Please show us a Q-Q plot, as well as mean, variance, skew and kurtosis of the transformed data for both labels. $\endgroup$
    – Wok
    Mar 13, 2011 at 22:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.