0
$\begingroup$

according to chapter 3 from 1-book, in case if there are discrete class labels, then the Gaussian likelihood is inappropriate. What are the exact reason for that and what would I reach if I use Gaussian likelihood despite of it (bad classification or would it be very poor).

Thanks!

cutout from the book:

enter image description here

1 - C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006,ISBN 026218253X.c©2006 Massachusetts Institute of Technology

$\endgroup$
1
$\begingroup$

First you need to know if your discrete variables are ordinal (there is a order into the labels) or not. If they are not ordinal you cannot expect a gaussian model to fit well the data since it will compare things which are not comparable.

If your discrete variable is ordinal and you have a lot then it is not clear that gaussian model is bad.

After you can also use "tricks" to get back to "gaussian" model. For example in binary classification you can try to estimate the probability of one class and not do a hard estimate (by trying to know which class exactly). You always can relax problems.

To conclude, your question is so wide that its all the answers i can give.

$\endgroup$
  • $\begingroup$ regarding variables, suppose I have MNIST classes, Y is discrete and ordinal. >>it will compare things which are not comparable. you mean by building the Cov due to kernel? $\endgroup$ – malocho Jul 24 at 14:57
  • $\begingroup$ >>If your discrete variable is ordinal and you have a lot then it is not clear that gaussian model is bad could you bring it nearer $\endgroup$ – malocho Jul 24 at 14:57
  • $\begingroup$ in Mnist its not really ordinal. You can compare label for sure but its just an artefact. Its useless to compare the label for the purpose of classification. I was speaking about discrete variable more like rating. Its a common discrete ordinal data. If you have a lot of labels for your ordinal data, its almost a continuous one. That was just my point, no more no less. $\endgroup$ – PauZen Jul 24 at 14:59
  • $\begingroup$ >>After you can also use "tricks" to get back to "gaussian" model. For example in binary classification you can try to estimate the probability of one class and not do a hard estimate (by trying to know which class exactly). You always can relax problems. I would classify directly without trick of binary classification $\endgroup$ – malocho Jul 24 at 15:00
  • $\begingroup$ regarding ordinality of Y, I haven't see anything in the book by Rasmussen, could you provide some sources, thank you $\endgroup$ – malocho Jul 24 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.