Problem in evaluating naive Bayes I am trying to understand naive Bayes and its application to text classification.
I have a doubt or this may be my misconception. 
Suppose we have two categories "News" and "Sports" in which we need to classify any given document. Let the dictionary contain only 3 keywords ${news,football,tennis}$ with the following parameters 
\begin{align*}
P(news/News)&=0.99,P(news/Sports)=0.01,P(tennis/Sports)=0.9,\\
P(tennis/News)&=0.1,P(football/Sports)=0.9,P(football/News)=0.1, \\
P(Sports)&=0.5 ,P(News)=0.5,P((news,football,tennis))=k;
\end{align*}
We get a document which has all the three keywords. So when we evaluate 
\begin{align}
P(News/(news,football,tennis))=0.99\cdot0.1\cdot0.1\cdot0.5/k=0.00495/k\\
P(Sports/(news,football,tennis))=0.01\cdot0.9\cdot0.9\cdot0.5/k=0.00405/k
\end{align}
So the document is classified to "News" category, but intuitively we  know that it should belong to "Sports" category. 
 A: Well: naive Bayes is called naive for a reason: the assumed conditional independence is often doubtful, even though it turns out to work well in a lot of practical cases.
Besides that: you have "chosen" your conditional probabilities so that it turns out this way. There is no (a priori) reason why P(tennis|News) and P(tennis|Sports) should sum to 1, but in this case this leads to the counterintuitive results.
A: A naive Bayes classifier, as the names suggests, is a simple application of Bayes' Theorem. Basically, it calculates the probabilities of quantities of interest (generally unobserved, called parameters or latent classes) based on the observed data. In your case the observed data are: news, football, and tennis. The quantities of interest for which you want to calculate the probabilities are: News and Sports. Seems like you are interested in calculating: $P(\text{News}|\text{news}, \text{football}, \text{tennis}), P(\text{News}|\text{news}, \text{football}, \text{tennis})$.
Now we will use Bayes theorem to get:
$$
P(\text{News}|\text{news}, \text{football}, \text{tennis}) = \frac{P(\text{news}, \text{football}, \text{tennis}|\text{News})P(\text{News})}{P(\text{news}, \text{football}, \text{tennis})}
$$
The first term in the numerator is calculated using the fact that given you observe the latent class, that is, News, the observed data, that is news, football, and tennis probabilities are independent (this may be a questionable assumption, but the answer depends on subject matter). You can use the law for calculating the probabilties of independent event.
$$
P(\text{news}, \text{football}, \text{tennis}|\text{News})=P(\text{news}|\text{News})P( \text{football}|\text{News})P(\text{tennis}|\text{News})
$$
Proceeding similarly for Sports, we get:
$$
P(\text{Sports}|\text{news}, \text{football}, \text{tennis}) = \frac{P(\text{news}, \text{football}, \text{tennis}|\text{Sports})P(\text{Sports})}{P(\text{news}, \text{football}, \text{tennis})} 
$$
$$
P(\text{news}, \text{football}, \text{tennis}|\text{Sports})=P(\text{news}|\text{Sports})P( \text{football}|\text{Sports})P(\text{tennis}|\text{Sports})
$$
The denominator for both cases can be calculated by using the Law of total probability. 
$$
P(\text{news}, \text{football}, \text{tennis}) =P(\text{news}, \text{football}, \text{tennis}|\text{News})P(\text{News})+ P(\text{news}, \text{football}, \text{tennis}|\text{Sports})P(\text{Sports}) 
$$ 
We are now left with only one probability in each case, that is,$P(\text{News})$ and $P(\text{Sports})$, respectively. If we know these, every probability until now can be calculated. This can be determined based on prior knowledge, or in your case it might be already provided to you. 
Plugging in all the probabilities gives you the probabilities of interest.  
A high probability value for a specific class implies that the observed document belongs to that class (News or Sports). But how do you decided "how high is high", depends, again, on subject matter and a lot of other issues. 
