1
$\begingroup$

I have a data set with following qualities:

  • Attributes: 500
  • Instances: 1500
  • Class Ratio: 1:15
  • Missing Values: Yes ~5%

How should I perform classification on this data which can give me best results.

The following things I have tried so far so far:

  1. Replaced missing values with mean (For time being, will be performing NA imputation using sophisticated methods)
  2. PCA (dimensionality reduction, reduced no. of columns from 500 to 30 with almost 90% data quality)
  3. Oversampling the minorities
  4. Ada boost algorithm for classification

Am I correct in my analysis? Although I am getting 100% accuracy overall and for both classes also.

Is there anyway I can oversample this data without doing PCA? I have tried SMOTE but that is throwing some colnames() error which I have no clue of.

Is there any package in R for any algorithm like ada boost, random forest, ipred which can take care of cost-sensitivity for such analysis? Or may be a tuning parameters for these algorithms by which I can set the cost parameter for minority class?

$\endgroup$
4
  • $\begingroup$ What do you mean by "attributes" and "instances"? It seemed like these would be equivalent to "variables" and "subjects" but then you used PCA to reduce "instances" so I am confused. $\endgroup$
    – Peter Flom
    Aug 9, 2013 at 10:47
  • $\begingroup$ @PeterFlom Sorry about the terminology. I meant Instances as cases(rows) and Attributes as variables(columns). Then I used PCA to reduce the columns from ~500 to 30. Does that clears your doubt? $\endgroup$
    – Ankit
    Aug 9, 2013 at 10:50
  • 1
    $\begingroup$ Thanks. You have a typo, then in that you say PCA reduced from 1500 to 30. You mean 500 to 30. $\endgroup$
    – Peter Flom
    Aug 9, 2013 at 10:54
  • $\begingroup$ @PeterFlom Oh sorry about that $\endgroup$
    – Ankit
    Aug 9, 2013 at 11:21

1 Answer 1

2
$\begingroup$

The desire for classification is causing problems for your analysis. Think seriously about building a probability (risk) model and using continuous accuracy scores. These will not mislead you, whereas optimizing classification accuracy will cause you to choose the wrong features, and classification accuracy (an improper scoring rule) has special problems with the prevalence of $Y=1$ in your sample is near 0 or 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.