3
$\begingroup$

In a recent exam on machine learning I came across the following question:

"Which of the following techniques can model the decision boundary depicted in the figure? (check all that apply)" See my self-made picture.

  • Logistic regression (with linear features)
  • Neural Networks
  • Naive Bayes
  • Support vector machine (with linear Kernel)

I was convinced the answer is Neural Networks and Naive Bayes. In particular Gaussian Naive Bayes can model circular decision boundaries (see an example here; it works, I also tried it myself).

However I have been told Naive Bayes was not rated as correct. Instead only neural nets was the correct answer.

How is this possible or is it a mistake?

enter image description here

Improve this question
$\endgroup$
7
  • $\begingroup$ Try SVM with Gaussian (RBF) Kernel or single linkage hierachical clustering. The 2nd suggestion is from this answer. $\endgroup$
    – Iliyan Bobev
    Commented Oct 31, 2016 at 19:27
  • $\begingroup$ This question seems perhaps more about the semantics of the test writer? My guess is there is supposed to be an implied "(with linear features)" after "Naive Bayes"? $\endgroup$
    – GeoMatt22
    Commented Nov 1, 2016 at 3:59
  • $\begingroup$ @GeoMatt22 okay, but there wasn't. $\endgroup$
    – tomka
    Commented Nov 1, 2016 at 6:47
  • 1
    $\begingroup$ wow, I never knew GNB can do that, I thought it is just a dumber version of a common linear classifiers. Even in sklearn examples page it is shown to be able to classify this circular problem scikit-learn.org/stable/auto_examples/classification/… and the explanation in the link you provided is quite convincing. I would say you were right $\endgroup$
    – rep_ho
    Commented Nov 18, 2016 at 21:55
  • 1
    $\begingroup$ @rep_ho yes, I had seen it somewhere and after the exam I went back to check it out. Turns out I was right in the end. $\endgroup$
    – tomka
    Commented Nov 18, 2016 at 22:54

1 Answer 1

Reset to default
1
$\begingroup$

You were right. Naive Bayes can actually create a circular decision boundary as the variance of red and black data points is different.

For Gaussian distribution estimation, if NB is forced to use identical variance within the class, then we cannot separate it well. Because then the NB classifier will fit only a linear hyperplane (line in this example).

I think your test writer has made the identical variance assumption implicitly while writing this question.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.