I've been learning machine learning through Andrew Ng's Coursera. I've completed Andrew's homework on SVM, but it felt wishy washed and I'm having hard time taking it from 0 to finish.
Say you calculated the SVM "model" using svmTrain provided by Andrew Ng. And now you want to see what the predicted output (using Andrew's svmPredict) is for all the training samples you used to calculate the error. Easy enough:
model = svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
predictions = svmPredict(model, Xval);
err = mean(double(predictions ~= yval));
All that make sense. What trips me up is what svmPredict is doing, specifically,
elseif strfind(func2str(model.kernelFunction), 'gaussianKernel')
% Vectorized RBF Kernel
% This is equivalent to computing the kernel on every pair of examples
X1 = sum(X.^2, 2);
X2 = sum(model.X.^2, 2)';
K = bsxfun(@plus, X1, bsxfun(@plus, X2, - 2 * X * model.X'));
K = model.kernelFunction(1, 0) .^ K;
K = bsxfun(@times, model.y', K);
K = bsxfun(@times, model.alphas', K);
p = sum(K, 2);
What equation/formula is it trying to solve here Gaussian Kernel (RBF)? What does the equation even look like?
Also, if I give you a new input $\vec u$, how would you find its predicted output?
The predicted output for the linear kernel looks pretty readable:
p = X * model.w + model.b;
Which is just solving $y=X\vec w+b$, with each example being a row in $X$.
Any help would be sincerely appreciated.