If I have aggregated statistics on individual words of a phrase, is there a way to compute the overall probability of a phrase having a "success" vs "not success" (binomial)?


Phrase: 8 inch wood lap siding

I have individual statistics on each word in this phrase.

  1. When 8 is in the phrase, "success" is 1/100 or 1%
  2. When inch is in the phrase, "success" is 1/200 or 0.5%
  3. When wood is in the phrase, "success" is 1/80 or 1.25%
  4. When lap is in the phrase, "success" is 1/125 or 0.8%
  5. When siding is in the phrase, "success" is 1/25 or 4.0%

What I'm trying to determine is if there is some mathematical way to compute the probability that a phrase will have a "success" if I know the individual probabilities of each word.

I don't know if this approach conceptually makes sense. Some words have high collocation with others so I don't know if this approach would capture that. Plus, some words appear very frequently (have heavier weights) and others don't.

  • $\begingroup$ Do you consider those probabilities to be independent? Do you have only information about marginal probabilities? If you have full dataset, that why not just simply use logistic regression? $\endgroup$
    – Tim
    Commented Nov 23, 2015 at 18:13
  • $\begingroup$ @Tim Not sure how to answer that. If I have 5 keywords, each keyword has a number of trials and successes. I split each keyword by space into words, then group by each word and sum trials and successes. A word would only appear in a keyword a max of 1 time but never twice or more. Ex: this is a keyword with 1/100 overall probability of "success" splits to this is a keyword but an important thing to note is that each word is assigned the overall probability of 1/100 equally which artificially creates more trials and "successes" when summed. Would logistic regression be a solution? $\endgroup$
    – Jarad
    Commented Nov 23, 2015 at 22:41

1 Answer 1


Of course there is.

For each keyword, you could happily compute the number of total times it succeeded and divide by the number of occurrences. You have now calculated the success for a given keyword.

But what happens when more than one keyword appears? You might simply multiply the probabilities. This would suggest you "naively" believe that every keyword is unrelated (hint: unlikely!), but may work nonetheless. This is what we'd call a Naive Bayes approach.

A more complex solution is grabbing all the words in a bag-of-words type model and feeding it into a more powerful classifier, or even creating "keywords" that are in fact ordered lists of words as found in the phrases (n-grams) or even the measure of occurrence of two words in the same statement (interaction features).

But there's no telling which is better unless you test it out on your data. What matters is what gives you better results.

What can be said is that the underlying assumptions for a Naive Bayes make it a simpler model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.