What is the difference between the probabilistic and non-probabilistic learning methods for different situations? When is one considered more appropriate than the other? Can you give me examples?


The task of classification enables a simple comparison:

A probabilistic approach (such as Random Forest) would yield a probability distribution over a set of classes for each input sample.

A deterministic approach (such as SVM) does not model the distribution of classes but rather seperates the feature space and return the class associated with the space where a sample originates from.

Yet it is possible for every probabilistic method to simply return the class with the highest probability and therefore seem deterministic. Further, the other way around, based on the distance to the seperating hyperplane in SVMs a probability can be computed and returned for each class.

Generally probabilistic approaches would be fit to incorporate incertainty regarding the answer and can provide information about how safe a prediction is. Further, probabilitic methods might be able to incorporate prior information about the class distribution.


If you would for example try to predict gender based on body heigth, a deterministic appraoch would simply chose a seperating point and say that:

(completely made up numbers)

height >= 175 --> man

height < 175 --> woman

so for a sample with the height of 175 it would yield woman as well as it would for a heigth of 155.

A probabilistic approach in this case could yield a probability such as for a height of

155 ---> man(0.1), woman(0.9) and for

175 --> man(0.49), woman(0.51)

  • $\begingroup$ The example was very useful, thank you for that. But if a classifier is using fixed set of rules in determining the class of an input (as in the deterministic approach) would it still be considered as a machine learning algorithm? I am new to ML, so apologies if this is not a correct question to ask $\endgroup$ – ansh sachdeva Apr 15 '20 at 21:25
  • $\begingroup$ The question is not whether the rules are deterministic or not but rather whether the rules were "learned" (~parameters adapted) from data. $\endgroup$ – Nikolas Rieble Apr 16 '20 at 4:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.