Are Machine Learning models probabilistic models? [closed]

Question

In terms of Machine Learning models, as far as I know, there are two kinds of models in Machine Learning. One kind are discriminative models and the other kind are generative models. But I wonder if those two kinds of Machine Learning models are probabilistic models.

Answer 1

Some, but not all, of the machine learning models, are probabilistic models. There are machine learning models that are probabilistic by design, such as Naive Bayes. There are also ones that are not probabilistic, like SVM, random forest, or $k$-NN, because they were not designed in terms of thinking of random variables and probability distributions.

On another hand, many models that are not defined in probabilistic terms per se, can be interpreted as probabilistic models. Minimizing a loss function is equivalent to maximizing some likelihood, for example minimizing squared loss is equivalent to maximizing Gaussian likelihood, and logistic loss to Bernoulli likelihood. Also regularization can be thought as setting priors for the parameters, so it is relatively easy to translate something like neural network, to Bayesian neural network. However, as you can learn from this thread, in some cases you can run into problems with it.

Being a probabilistic model is a "nice to have" feature for machine learning model, because it makes easier for us to evaluate the uncertainties related to the predictions returned by the model. It is not "obligatory", because in machine learning we usually worry only about making accurate predictions (rather then inference), and in many cases the well established tools for assessing their performance, like cross-validation, are enough for this task. For example, $k$-NN classifies cases based on their similarities to other samples in the training data, and this both works and is intuitively appealing way of making classifications, there is no need to defining it in probabilistic terms.