1
$\begingroup$

I have a fairly simple data set that I gathered from running workloads on a Hadoop Cluster. My goal is to model the running times of this application based on the feature variables of interest. The full dataset is available

Kmeans Report Data Set

My full R code is given below

library(ggplot2)
library(dplyr)
library(gridExtra)
library(caret)
library(ggpubr)
library(e1071)
library(caTools)

allDataOriginal <- read.table(file = "kmeans.report", header = TRUE)

allDataOriginal$DataSizeMB <- round((allDataOriginal$Input_data_size/1048576))

allDataOriginal <- select(allDataOriginal, Duration.s., NumEx, ExCore, ExMem, LevelPar, DataSizeMB)

#Do some data preprocessing

allDataOriginal$ExMem = as.integer(gsub("g", "", allDataOriginal$ExMem))
head(allDataOriginal)
str(allDataOriginal)
allDataOriginal = filter(allDataOriginal, allDataOriginal$Duration.s. <= 3000)

set.seed(123)
split = sample.split(allDataOriginal$Duration.s., SplitRatio = 0.8)
training_set = subset(allDataOriginal, split == TRUE)
test_set = subset(allDataOriginal, split == FALSE)

#Build the model
fit = svm(Duration.s. ~ DataSizeMB + NumEx  + ExCore, data=training_set, type = 'eps-regression')

#Plot to visualise
ggplot(test_set) +
  geom_point(aes(seq(1:nrow(test_set)), Duration.s.), color='red')+
  geom_line(aes(seq(1:nrow(test_set)), predict(fit, test_set)), color='green')+
  ggtitle("SVR Model")+
  xlab("Index")+
  ylab("Time")

Now the problem is when I try to predict extreme value which I know should give me larger running time, I always get an incorrect output because I know the values should be greater than that. Example the model returns the same value for all the four predictions below

predict(fit, data.frame("DataSizeMB" = 40000, "NumEx" = 4, "ExCore" = 8, "ExMem" = 16))
predict(fit, data.frame("DataSizeMB" = 400000, "NumEx" = 4, "ExCore" = 8, "ExMem" = 16))
predict(fit, data.frame("DataSizeMB" = 40000, "NumEx" = 4, "ExCore" = 2, "ExMem" = 2)) 
predict(fit, data.frame("DataSizeMB" = 400000, "NumEx" = 4, "ExCore" = 2, "ExMem" = 2))

all returns the value 171.4604

I am following a course on Udemy and their code is similar to what I have but the datasets are not the same. What am I missing?

I am new in this area so I may be missing some fundamentals.

$\endgroup$
2
$\begingroup$

I think your problem are the parameters of your SVM-modell.

In your example you converge to 176. You can see this by plotting with:

val = 0
for (i in 1:400){
  val<- c(val,predict(fit, data.frame("DataSizeMB" = i*100, "NumEx" = 4, "ExCore" = 8, "ExMem" = 16)))
}
plot(val)

plot1

You need to change your gamma values and the cost. In the tutorials they often take easy data samples, but adjusting them is not an easy task. From the python scikit documentation:

When training an SVM with the Radial Basis Function (RBF) kernel, two parameters must be considered: C and gamma. The parameter C, common to all SVM kernels, trades off misclassification of training examples against simplicity of the decision surface. A low C makes the decision surface smooth, while a high C aims at classifying all training examples correctly. gamma defines how much influence a single training example has. The larger gamma is, the closer other examples must be to be affected.

.In you example i achieve with gamma = 0.001 and cost = 1000 more reasonable values:

fit = svm(Duration.s. ~ DataSizeMB + NumEx  + ExCore, data=training_set, type = 'eps-regression', gamma = 0.001, cost= 1000)

output: plot2

But then another problem occurres. Your values are completly dependent of your datasize, if you try using other parameters, they is nearly no influence. This happens, since the values are in completly different dimensions (check here)

Finding correct gamma and cost values is not an easy task. You can try using GridSearch functions (i think for prediction it's GridSearchCV) or adjust it manual. Also try to understand the SVM model and how your data behaves in your Model, since this can bring other problems, like i mentioned above.

$\endgroup$
  • $\begingroup$ Okay, this is a good pointer. I will have a proper look at adjusting the gamma and cost values of the SVM model. As for the normalisation, can't I just use the scale on the data frame? $\endgroup$ – fanbondi May 8 at 9:58
  • $\begingroup$ @fanbondi it strongly depends on how you want to weight your data. Like i said, in your example if you use 400000 for DataSizeMB a 2 or 8 doesnt matter anymore for cores. Normalisation can achieve that you weight everything the same. But also cores can have an exponential influence. So it's up to you how to use your dataset. Just be aware, that SVM doesn't interprete your values. It's just math, so a distance between 0 and 40000 ist much higher then 2 to 4, but you can also use different distance functions (check the link). A general solution for your problem can not be easily given $\endgroup$ – mischva11 May 8 at 10:04
  • 1
    $\begingroup$ @mischava11, that makes a lot of sense. I will dig deeper to see how I can generalise or have the best case in my scenario. $\endgroup$ – fanbondi May 8 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.