3
$\begingroup$

There are some resources online (e.g. this one) on logistic regression with polynomial kernels, such as

$$h_\theta(x)=logistic(\theta_0 + \theta_1x1+ \theta_3x_1^2 + \theta_4x_2^2)$$

I'm wondering if it's possible to use RBF kernel $K_r(x,x')=\exp(-\frac{\|x-x'\|^2}{r})$ here. If so, what would $h_\theta(x)$ looks like?

$\endgroup$
1
  • 4
    $\begingroup$ It's not kernel. You just add nonlinear transformations of input variables to your existing input variables. $\endgroup$
    – seanv507
    Dec 10, 2015 at 7:59

1 Answer 1

1
$\begingroup$

For others who have similar questions, I found this paper by Zhu and Hastie (doi:10.1198/106186005X25619, JSTOR, Citeseer$^\dagger$) very helpful.

By the representer theorem, $f_\theta(x)$ has the form:

$$f_\theta(x)=b + \sum_i ^ N a_i K(x, x_i) $$

So the probability is $p(x)=\frac{e^{f(x)}}{1+e^{f(x)}}$.


$^\dagger$ The link doesn't work.

$\endgroup$
3
  • $\begingroup$ @user777 Thanks. I think I'm asking for RBF kernel. $\endgroup$
    – qweruiop
    Jan 15, 2016 at 21:27
  • $\begingroup$ @user777 Of course it's not. It's nothing. It's just a quote from the resource I linked. $\endgroup$
    – qweruiop
    Jan 15, 2016 at 21:31
  • 1
    $\begingroup$ For efficient tuning of the hyper-parameters, see my paper (with Mrs Marsupial) "Efficient approximate leave-one-out cross-validation for kernel logistic regression" link.springer.com/article/10.1007%2Fs10994-008-5055-9 (pre-print available here: theoval.cmp.uea.ac.uk/publications/pdf/ml2008a.pdf ). We do use the RBF kernel in that paper. KLR is a very useful machine learning algorithm. $\endgroup$ Jan 7, 2022 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.