7
$\begingroup$

I am reading Chris Bishop's Pattern Recognition and Machine Learning textbook. I came across the term probabilistic inference several times. I have a couple of questions.

  1. Is probabilistic inference only applicable in a graphical modelling context?

  2. What's the distinction between traditional statistical inference (p-values, confidence intervals, Bayes factors etc.) and probabilistic inference?

  3. Is this a term that's specific to the CS community or is it widely used in the statistics community as well?

$\endgroup$
  • 3
    $\begingroup$ In my opinion, this is just a fancy denomination (and an oxymoron) that reproduces the fact that statistics is based on probabilistic modelling. $\endgroup$ – Xi'an Nov 2 '16 at 12:45
  • 1
    $\begingroup$ Thank you @Xi'an, I will continue to use statistical inference in my paper and presentations. $\endgroup$ – discretetimeisnice Nov 3 '16 at 10:19
8
$\begingroup$

Probabilistic inference uses probabilistic models, i.e. models that describe the statistical problems in terms of probability theory and probability distributions. While statistics use probability theory quite heavily, you cannot say that those two disciplines are the same thing (check the discussion in this thread). Notice that many statistical and machine learning methods do not explicitly use probability theory to define the problems, e.g. many clustering algorithms, or classification methods that work by minimizing some loss function etc. But the distinction is not that straightforward, take as example approximate Bayesian computation -- theoretically it is based on Bayesian (probabilistic!) inference, but it deals with cases where we do not have likelihood function, so instead of it we use a distance measure.

$\endgroup$
  • $\begingroup$ (+1): I feel less confident about the distinction as I would not classify statistics outside probabilistic modelling. Given that an essential tenet of statistics is to include an assessment of uncertainty, and that this uncertainty need be modelled by a probabilistic structure, I do not see how statistics escapes this framework. $\endgroup$ – Xi'an Nov 2 '16 at 12:12
  • $\begingroup$ I also object to the ABC link. Although it is nice to see it mentioned, ABC is on the opposite a good illustration of probabilistic modelling as it replicates the exact production of a random sample according to a given probabilistic model! That the likelihood function cannot be numerically computed does not mean that the probability model does not exit, only that it has to be considered from another perspective. $\endgroup$ – Xi'an Nov 2 '16 at 12:14
  • $\begingroup$ @Xi'an I agree with your comments, but if one wants to make such distinction between probabilistic and non-probabilistic models, then it could be defined as above. Nonetheless, the distinction is quite abstract, ambiguous and in many cases useless, for the reasons you gave. As about ABC, it is just an illustration of case where things get blurry and the distinction becomes ambiguous. $\endgroup$ – Tim Nov 2 '16 at 12:17
  • $\begingroup$ "classification methods that work by minimizing some loss function", can you provide an example of such a method that couldn't be formulated as a probabilistic model? $\endgroup$ – nbro Jan 16 at 16:50
  • $\begingroup$ @nbro I said "do not use explicitly...", not that they cannot be formulated like this. $\endgroup$ – Tim Jan 16 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.