After training a Gaussian Process (GP), predictions for any input is the mean a probability distribution (posterior) and the uncertainty in making this prediction is the variance of this posterior. However, in other ML models (like neural networks) we have no direct way to extract uncertainty (an indirect way would be to use dropout when making predictions, see here).
My question is: is there a clear/agreed-upon terminology to differentiate between models that offer a direct way to compute uncertainty (like GPs) and those that don't (neural networks)? Intuitively, I referred to models like GPs as probabilistic models (since predictions are probability distributions) and models like neural networks as non-probabilistic. However, a friend of mine objected saying that a neural network employing stochastic gradient descent in training is also a probabilistic model.
