1
$\begingroup$

In Naive Bayes Why is it necessary for Naive to assumes that the input features are independent and not co-related .

can anyone explain with a very simple example on what is the problem of events being dependent in Bayes therom ( NON-Naive Bayes in this case ) , why is that its become a global rule to applying Naive and make the events being independent ?

please explain with a simple example in layman term the difference between Naive Bayes and Non-naive Bayes.

if we have a sentence "You won lottery for 1million" and we need to classify it as spam and not spam .

in the likelihood part we will model the probability as p(x|y)

here x="You won lottery for 1million" and y=spam or not spam

p('You won lottery for 1million'|y=spam)

p('You won lottery for 1million'|y=notspam)

why is it so hard to calculate above probability that we need naive to pitch in and make the features independent to calculate the probability ,

when using independence if any one of the probablity of event is 0 then it make the who probability zero right ?

p(you|y=spam)* p(won |y=spam)*p(lottery|y=spam)*p(for|y=spam)*p(1million|y=spam)

p(you|y=notspam)* p(won |y=notspam)*p(lottery|y=notspam)*p(for|y=notspam)*p(1million|y=notspam)

Edited the question where i gave a example myself .

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

As you note, Naive Bayes assumes the input features (predictors) are not correlated.

This is a "naive" assumption, because input features commonly are correlated, just as regression predictors can be correlated (the problem of multicollinearity). But in some situations Naive Bayes models can work reasonably well and are much simpler to calculate.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ +1 Indeed, the naïve Bayes model works best when the covariance is diagonal. The naïve Bayes model becomes weaker and weaker as the matrix becomes increasingly non-diagonal, i.e. the features become more (anit-)correlated, because the model is increasingly in conflict with reality. $\endgroup$
    – Sycorax ♦
    Commented Feb 18, 2020 at 15:50
  • $\begingroup$ @zbicyclist I have give an example myself and addede to the question now ,can you please look into it as im not able to understand your current explanation as i want to know whats the issue with being non independent (non naive) it would be great if can you exaplin with a same example or ascenario or point to any links that i can read more about it . $\endgroup$
    – star
    Commented Feb 18, 2020 at 18:42
  • $\begingroup$ @Sycorax says Reinstate Monica I have give an example myself and added to the question now ,can you please look into it as im not able to understand your current explanation ,how do i compute the matrix and check if covariance is diagonal for the spam/notspam example that i have shared in the question . $\endgroup$
    – star
    Commented Feb 18, 2020 at 18:45
0
$\begingroup$

Because Naive Bayes assumes your features are not correlated, you don't need to provide an explicit a priori causal model - or rather, you already have, just implicitly.

A Naive Bayes algorithm for predicting cancer would assume $p(Cancer|Smoking,Tar) = p(Cancer|Smoking)p(Smoking) + p(Cancer|Tar)p(Tar)$

A Hierarchical Bayesian model could specify a model - Tar->Smoking->Cancer, for example, and use it to construct an equation $p(Cancer|Smoking,Tar) = p(Cancer|Smoking)p(Smoking|Tar)p(Tar)$.

Note that this model is A) simple - you only have 3 nodes, and the number of possible vertices grows rapidly, and B) deliberately wrong - it gets the cause and effect between smoking and lung damage backwards.

Depending on the interactions in the real world, getting the dependencies wrong may be a far worse problem than just assuming none exist, and even if you do get it right, a hierarchical model is both less straightforward and more computationally intensive to fit.

If you have a nice, fairly clean set of independent features Naive Bayes gets around all that, but on the flipside a good Hierarchical Bayes model is really powerful.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Please note i have added a spam/not spam example in the question can you please look into it and give me an reson for why naive is needed ? $\endgroup$
    – star
    Commented Feb 18, 2020 at 18:47
  • $\begingroup$ p(Cancer|Smoking,Tar)=p(Cancer|Smoking)p(Smoking)+p(Cancer|Tar)p(Tar) .Are you talking about the naive part or bayes part ? i guess its the bayes part as you trying to find disease given set of habits and not the set of habits given diseases so you should have writen it as p(Cancer|Smoking,Tar)=p(Smoking|cancer )*p(Tar|cancer ) right ? , im confused on your probability example of Naive Bayesitself can you please explain this probablity first and how you got p(Cancer|Smoking)p(Smoking) this conditional independence dose not make any logic to me . $\endgroup$
    – star
    Commented Feb 18, 2020 at 19:12
  • $\begingroup$ Re 1: it's not needed, but properly specifying the correlations would require an industrial-strength language model, and likely would not perform that much better incost/benefit in many applications. $\endgroup$
    – jkm
    Commented Feb 18, 2020 at 22:47
  • $\begingroup$ Re 2: this is a simple application of the chain rule p(A,B)=p(A|B)p(B) and law of total probability under the assumption that Smoking and Tar are independent of each other (i.e. p(Smoking|Tar)=p(Smoking) and vice versa) and Cancer depends on them both. It's not specific to either Bayesian model, it's just basic conditional probability/set theory manipulation. $\endgroup$
    – jkm
    Commented Feb 18, 2020 at 23:02
  • $\begingroup$ -> but can you answer my main question ,im not able to corelate to your example as its confusing , My query is simple , Naive -> helps to calculate probabality considering independence of event Non-Naive -> its the opposite of naive right ,where we calculate probability considering NON independence of event how do you write and find the probability for Non-Naive? this is my simple question ,on what basis the naive is advantageous over non naive ? $\endgroup$
    – star
    Commented Feb 19, 2020 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.