Document classification question I've got a tricky document classification question.  Each document in my collection can be assigned a label.  In my training set the average is about 3 different labels assigned to each document.  What I have learned in the training set is $P(\text{label}|\text{doc_word})$ for many label/doc_word pairs.  I want to calculate
$P(\text{label}|\text{word}_1,\ldots,\text{word}_n)$ but I cannot simply calculate it by
$$P(\text{label}|\text{word}_1,\ldots,\text{word}_n)=P(\text{label}|\text{word}_1)\cdot \ldots \cdot P(\text{label}|\text{word}_n)$$
because there are many words where $P(\text{label}|\text{word}_i)=0$.  Furthermore, if there are two words in the document where $P(\text{label}|\text{word})>0$, their product would be less than the individual probabilities thus making the label less likely.  This is counter-intuitive to me because I would think that having two words where $P(\text{label}|\text{word})>0$ should make the label more likely.  So maybe the independence assumption does not hold.
Perhaps I should just use these probabilities as association rules in order to assign labels?
 A: This is a really interesting question; I'm assuming that you are using a maximum likelihood framework, where the likelihood of of a particular word $w_i$ appearing in a document having label $L$ is simply given by $p(L|w_i) = \frac{count(L,w_i)}{count(w_i)}$.
Now, part of the reason that this problem arises is because of the use of this maximum likelihood framework, where the likelihood that a document has label $L$ can only be non-zero when all of the words in the constituent document have appeared at least once in documents with label $L$ in our observations before. In that sense, your derivation is correct; when working with a maximum likelihood framework, it is entirely reasonable to assign a document zero probability of having label $L$ even if only one of its words has never had label L before.
That said, this is obviously quite unsatisfactory for our purposes. One, on a practical level, it is rather evidently not true. And two, it rather seriously affects our ability to score and classify documents based on the evidence. I'll talk about each one briefly.
The first objection - that is, that on a practical level it doesn't really work as we've formulated - is entirely the result of the fact that the maximum likelihood approach is a frequentist one. In a Bayesian treatment, these probabilities might be very small, but would presumably be non-zero as the result of the inclusion of some reasonable prior.
Despite that caveat, can still use this framework for prediction, bringing us to number two. One approach is to do what you mentioned; that is, to entirely discard the probabilistic framework and simply use the observed counts as scoring features, allowing ranking of particular labels for a given document. In this formulation, simply label a given document $D$ via the procedure:
$label(D)= arg max_{l \in L}[\sum_{w_i \in D} P(l|w_i)]$
This is nicely computationally tractable, of course, and uses the counts we've already gathered. Note, by the way, that despite the use of the observed MLE probabilities for scoring, the quantity yielded by this is NOT normalized, and thus not a probability distribution.
However, this approach does have shortcomings - for example, it does not deal with the fact that the probability distribution over the labels in $L$ is likely not uniform. There are a number of ways to treat this; a commonly used approach which would be useful here is the Naive Bayes Classifier, which nicely adds estimates over the label observations. 
Lastly, given the pairs that you have, and the number of irrelevant features that are appearing in each document (ie, words that have no real use in classification, like "the"), you might look into linear models such as Winnow, which is extendable to multiple classes and is designed for dealing with large numbers of irrelevant features.
I hope that helps!
A: Let me clarify a couple of things.
Fist of all, when you use Naive-Bayes to calculate the probability  $P(\text{label}|\text{doc_word})$, you have to change that relation into something you know. In this case, the natural route is to use Bayes' theorem that states the following: 
$$P(\text{label}|\text{doc_word}) = \displaystyle \frac{P(\text{doc_word}|\text{label})P(\text{label})}{P(\text{doc_word})}$$
Usually, you can disregard the denominator because it only provides a normalization factor, which is usually not needed.
As you said, there are issues regarding the calculation of $P(\text{doc_word}|\text{label})$ when you don't have a word assigned to a given label, so the whole expression would be zero. However, there is an easy fix. You can pretend that words not assigned to a given label are actually seen once. Therefore, the maximum likelihood estimate, that is, the good old frequentist view of probability, is calculate as:
$$P_{Lap}(\text{word}|\text{label}) =\frac{ C(\text{word}) + 1}{N + d}$$
where $C(\text{word})$ is the number of times a given word appears assigned in a label, $N$ is the total number of words in your corpus and $d$ is a correction factor that accounts for the number of extra words you are considering in the corpus.
This trick has a serious drawback. As you should expect, there are a fairly large number of words you may encounter in a document that are not assigned to any label, so you are allocating a tiny bit of probability to those words and subtracting the same tiny bit to the words in your corpus. Therefore, a large chunk of the probability space will be been given to unseen words, and that is quite unsatisfactory. That is why there have been proposal to remedy this issue. One of them is the  Lidstone smoothing, in which we add a fraction $\lambda$ instead of $1$:
$$P_{Lid}(\text{word}|\text{label}) =\frac{ C(\text{word}) + \lambda}{N + d\lambda}$$
An even better estimate of the frequency of a word is the Good-Turing estimator, so I encourage you to give it a read.
Depending on your particular application, you may want to try more sophisticated smoothing procedures, but this is really the low-hanging fruit. 
