1
$\begingroup$

I am trying to show the decision surface of an SVM model and its margins in a 3D RGL graph in R. Here is a really nice, replicable example from stack overflow. I am using the same example:

library(e1071)
library(rgl)
library(misc3d)

n    = 100
nnew = 50

# Simulate some data
set.seed(12345)
group = sample(2, n, replace=T)
dat   = data.frame(group=factor(group), matrix(rnorm(n*3, rep(group, 
each=3)), ncol=3, byrow=T))

# Fit SVM
fit = svm(group ~ ., data=dat, kernel = "linear")

# Plot original data
plot3d(dat[,-1], col=dat$group)

# Get decision values for a new data grid
newdat.list = lapply(dat[,-1], function(x) seq(min(x), max(x), len=nnew))
newdat      = expand.grid(newdat.list)
newdat.pred = predict(fit, newdata=newdat, decision.values=T)
newdat.dv   = attr(newdat.pred, 'decision.values')
newdat.dv   = array(newdat.dv, dim=rep(nnew, 3))

# Fit/plot an isosurface to the decision boundary
contour3d(newdat.dv, level=0, x=newdat.list$X1, y=newdat.list$X2, 
z=newdat.list$X3, color="orange" add=T)

One can move the drawn surface around by adjusting the level argument. I created two additional semi-transparent margins at level=-0.2 and level=0.2, with alpha=0.1 for the semi-transparency:

contour3d(newdat.dv, level=-0.2, x=newdat.list$X1, y=newdat.list$X2, 
z=newdat.list$X3, add=T, color="gray", alpha=0.1)
contour3d(newdat.dv, level=0.2, x=newdat.list$X1, y=newdat.list$X2, 
z=newdat.list$X3, add=T, color="gray", alpha=0.1)

My semi-transparent margins now trace the points where the decision values are equal to -0.2 and 0.2 respectively. However, I chose these numbers arbitrarily, just to see if my code works. What would be the proper way of calculating the decision values exactly at the margins?

I am very new to machine learning and have no math or engineering background at all. I get completely lost in the formulas.

EDIT: To make my question a little more straightforward, I forced a linear kernel

$\endgroup$
0
$\begingroup$

The most common form of SVM (C-SVM) has the values at the margin as 1 and -1. Of course, due to numerical (in)stability, support vectors on the margin may not have precisely that value, but should within some small $\epsilon$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.