I have basic understanding of Stats and till now have worked with Linear & Logistic Regression, Random Forests etc. Introduction to Statistical Learning was my go to book.I never worked with or studied Bayesian Statistics and was blissfully unaware of the whole Frequentist vs Bayesian thinking/debate.

Now I am working on a new project using Bayesian networks and I am really struggling to grasp the concepts. My main issue is that when looking through the different modelling techniques I found that all things like Regression, Decision trees etc are also done through Bayesian Statistics. This made me realize that Bayesian statistics is not just about new techniques but a way of rethinking everything I studied before.

Can someone provide a simple road map of the different techniques under both kinds of stats and when to use which.

I have not studied Statistics formally but now working in machine learning I am learning as I go. Bayesian Statistics looks daunting.

  • 1
    $\begingroup$ Did you read "Bayesian reasoning and Machine Learning"? It's available online for free $\endgroup$ – Jakub Bartczuk Sep 11 '17 at 9:01
  • $\begingroup$ I just downloaded... thnx for the pointer $\endgroup$ – pankaj negi Sep 11 '17 at 9:07
  • 1
    $\begingroup$ As a quick comment I would say that you can always use bayesian methods: if you want to predict a continuous quantity y given some data x then using a bayesian model will give you the "full" distribution p(y|x) and you can turn that into a very concrete prediction for y by taking its mean. It is somewhat desirable to look at p(y|x) because not only does it give you a hint what value y could attain but it also gives you the uncertainty with which you predict this y, i.e. if p(y|x) is "spiky" around its mean then you are pretty confident and if it is flat then you are rather uncertain... $\endgroup$ – Fabian Werner Sep 11 '17 at 9:30
  • 1
    $\begingroup$ However, one should mention that this comes at the cost of modeling a prior p(Theta) for the parameters and one or more conditional distributions p(x|Theta) and this is something you need to justify and you could easily screw up your model (i.e. it happily predicts the wrong values with greatest confidence) by selecting the "wrong" prior or the wrong conditional distribution. On the other hand, once you have chooses the main distribution there is a canonical choice for the prior, namely its conjugate (see en.m.wikipedia.org/wiki/Conjugate_prior) $\endgroup$ – Fabian Werner Sep 11 '17 at 9:35
  • $\begingroup$ "Can someone provide a simple road map of the different techniques under both kinds of stats" seems far too broad (of the 'takes several books to answer' kind). Even just a list of different techniques under frequentist statistics would be too broad. $\endgroup$ – Glen_b Sep 11 '17 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.