# What is the difference between the probabilistic and non-probabilistic learning methods? [duplicate]

What is the difference between the probabilistic and non-probabilistic learning methods for different situations? When is one considered more appropriate than the other? Can you give me examples?

The task of classification enables a simple comparison:

A probabilistic approach (such as Random Forest) would yield a probability distribution over a set of classes for each input sample.

A deterministic approach (such as SVM) does not model the distribution of classes but rather separates the feature space and return the class associated with the space where a sample originates from.

Yet it is possible for every probabilistic method to simply return the class with the highest probability and therefore seem deterministic. Further, the other way around, based on the distance to the separating hyperplane in SVMs a probability can be computed and returned for each class.

Generally, the probabilistic approaches are better suited to incorporate uncertainty regarding the answer and can provide information about how safe a prediction is. Further, probabilistic methods might be able to incorporate prior information about the class distribution.

Example:

If you would for example try to predict gender based on body height, a deterministic approach would simply chose a separating point and say that:

height >= 175 --> man

height < 175 --> woman

so for a sample with the height of 175 it would yield woman as well as it would for a height of 155.

A probabilistic approach in this case could yield a probability such as for a height of

155 ---> man(0.1), woman(0.9) and for

175 --> man(0.49), woman(0.51)

• The example was very useful, thank you for that. But if a classifier is using fixed set of rules in determining the class of an input (as in the deterministic approach) would it still be considered as a machine learning algorithm? I am new to ML, so apologies if this is not a correct question to ask Commented Apr 15, 2020 at 21:25
• The question is not whether the rules are deterministic or not but rather whether the rules were "learned" (~parameters adapted) from data. Commented Apr 16, 2020 at 4:19
• @NikolasRieble Since all models can turn probabilities into predicted classes and scores into probabilities, what is the fundamental distinction? I would argue (I am not sure) that in the probabilistic inference we directly model a probability distribution whereas in non-probabilistic we just learn decision boundaries. Commented Jun 10, 2023 at 14:09