# Decision tree: Perfect classification with (a dicotomic) class noise at 100%, but almost null prediction with noise at 99%. (Tried 2 alg in R). Why?

I am using a dataset with a dicotomic class and testing how noise affects the decision tree j48 - from Rweka, using R - performance. I´m adding noise, and using confindence factors from 0.01 to 0.5 along the experiment.

As the class is dicotomic, it makes perfect sense that the worst performance of the training set is at 50%, being the class dicotomic.

Now... as the validation goes down, I´ve some points where the accuracy is almost perfect when the noise is total. How can I explain that?

Red is for the training set; Blue for the validation (80/20)

accuracy = (correctly predicted class / total testing class) × 100% OR, The percentage of correctly classified instances (TP + TN)/(TP + TN + FP + FN). where TP, FN, FP and TN represent the number of true positives, false negatives, false positives and true negatives, respectively.

The dataset I´m using is a random sample (10%) of this one https://www.kaggle.com/ludobenistant/hr-analytics

Here´s my code:

library(arules)
library(RWeka)
library(caret)
library(caTools)

setwd("C:\\Users\\Lucas\\Desktop\\AA")
inputFileName = "hr.csv"

ruido = function( datasetName, percentage ) {

sample = sample.split(hr$left, SplitRatio = percentage / 100) toModify = subset(hr, sample == TRUE) NotModify = subset(hr, sample == FALSE) toModify$left = sapply(toModify$left, function(x) !x ) new_hr = rbind(toModify, toNotModify) output = strsplit(datasetName, "[.]") outputFileName = paste0(output[[1]][1], '_ruido_', percentage, '.', output[[1]][2]) write.csv(new_hr, file = outputFileName, row.names=FALSE) outputFileName } set.seed(101) percentages = seq(0, 100 ) confidenceFactors = seq(0.05, 0.5, 0.05) dataset = read.csv( inputFileName, sep = "," ) sample = sample.split(dataset$left, SplitRatio = .8)
train = subset(dataset, sample == TRUE)
test  = subset(dataset, sample == FALSE)

dir.create(file.path("C:\\Users\\Lucas\\Desktop\\AA", "datasets\\5"))
write.csv(train, file = "datasets\\5\\HR_train.csv", row.names=FALSE)
write.csv(test, file = "datasets\\5\\HR_test.csv", row.names=FALSE)

sizeResults = data.frame( "CF" = double(), "Percentage" = double(), "Value" = integer(), stringsAsFactors=FALSE)
accuracyResults = data.frame( "CF" = double(), "Method" = character(), "Percentage" = double(), "Value" = integer(), stringsAsFactors=FALSE)

for( percentage in percentages ){

percentage = percentage / 100

outputFile = ruido( "datasets\\5\\HR_train.csv", percentage * 100 )
training$number_project = as.factor( training$number_project )
training$time_spend_company = as.factor( training$time_spend_company )

for( c in confidenceFactors ){
# create tree
print( paste0("Decision tree for: ", "percentage=", percentage, ", CF=", c) )
tree = J48( as.factor(left) ~ ., training, control = Weka_control(M=2, C=c) )
treeSize = tree$classifier$numElements()
sizeResults = rbind(sizeResults, data.frame(CF=c, Percentage=percentage, Value=treeSize))

# test tree
pred <- predict(tree, test, type='class')
trainingAccuracy = summary(tree)$details[[1]] testAccuracy = confusionMatrix(table(test$left, pred))
accuracyResults = rbind(accuracyResults, data.frame(CF=c, Percentage=percentage, Method="Entrenamiento", Value=trainingAccuracy))
accuracyResults = rbind(accuracyResults, data.frame(CF=c, Percentage=percentage, Method="Validaci�n", Value=testAccuracy\$overall[[1]] * 100))
}
}

ggplot(data=sizeResults, aes(x=Percentage, y=Value)) +
geom_point() +
ylab("Nodes") + # Set axis labels
scale_colour_hue(name="")       # Set legend title

ggplot(data=accuracyResults, aes(x=Percentage, y=Value, group=Method, color=Method)) +
geom_point() +
ylab("Accuracy") + # Set axis labels
scale_colour_hue(name="")

• So what is the bottom axis? The percentage of the dataset that is represented by random noise?? – Josh Jun 14 '17 at 2:54
• @Josh it´s the noise % in the class. The class is dicotomic, so a 100% noise means all the class labels got inverted. – Luis Jun 14 '17 at 3:00
• I see... And now I see you're referring to that single blue dot with the arrow. Wow! – Josh Jun 14 '17 at 3:10
• Yes. Do you know what that may be? – Luis Jun 14 '17 at 3:12
• no clue! This feels like more of a coding question and might do better in stack overflow – Josh Jun 14 '17 at 3:28