I try to get used to some classification methods in R (kNN, Decision Trees, SVM) and I am just wondering: Is there a way to do a biased random guess classification to see the real performance of the classificator?

Update: Example: There are two classes, but its a imbalanced data set. Class 1 makes 70 %, class 2 30 %. Therefore, its not a big deal do "guess" 70 % correct by classifying each data record as class 1. So I want to show the following: Classificator: 90 % identified as TP random guess (biased, for the known distribution): 73 %

The random guess should just identify the data records by guessing. If the distribution would be balanced, it would show approx. a 50/50 result. With the known distribution it would show a approx. 70/30 result. Hopefully this clarifies the question a little bit...



migrated from stackoverflow.com Jan 25 '15 at 20:24

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ When you post a general question with no code, people are going to see it as a statistical question rather than one that asks for a specific coding solution. $\endgroup$ – DWin Jan 25 '15 at 19:39
  • 1
    $\begingroup$ It is not clear what you are asking for - but you can imagine a chimp who is going to be replaced by your code; how many classes do you have in your case ?! $\endgroup$ – user4581 Jan 25 '15 at 22:36
  • $\begingroup$ Thank you for this hint. I added a update, hopefully its better now. $\endgroup$ – Stefan Jan 27 '15 at 12:12

In Matlab you'd do this for a single observation (assuming a binary classification):

p = 0.7;
y = rand() > p;

or if your input set of binary labels is a vector Y then you can do:

p = mean(Y);
y = rand(size(Y)) <= p;

I imagine the R code would be nearly identical.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.