2
$\begingroup$

I read a book recently and it mentioned related to prior probability of naïve bayesian:

"Since the probability of an outcome is calculated from the data set, it is important that the data set used for data mining is representative of the population, if sampling is used. A class-stratified sampling of data from the population will not be compatible for naïve Bayesian modeling." -- Page 114-115 and the book-website friendly offers entire eBook view.

Within the context of the book, the first term of probability (in italic font style) means the prior probability of certain outcome $P(Y)$, and the two times of "data set" mean training data set, from my understandings. Let's say we have training data for naïve bayesian modeling:

Example Training Data 1 id credit income default-on-mortgages 1 100 $100 yes 2 1000 $1000 no 3 50 $50 yes

Prior probability of $P(Y=yes)=2/3$, $P(Y=no)=1/3$.

I cannot figure out why stratified sampling does not suit for bayesian modeling?

$P(Y)$ is one of the essential probabilities to solve questions. The essence and smartness of naïve bayesian are computing the prior probability $P(Y)$ and class conditional probability $P(X\mid Y)$ and they will lead to result of the unknown $P(Y\mid X)$. The former two probabilities are relatively easier to get compared with the latter one.

If sampling method is used, I will expect the $P(Y)$ of subset will be close to $P(Y)$ of the original training set. So that the training results of applying NB algorithms on two data sets would not be much different. I assumed, stratified sampling could ensure subset and original data set have similar class contents, for example, the original data set has 150 samples with 100 "default=yes" and 50 "default=no". In this case, $P(Y=yes)=100/150=2/3$. If I used stratified sampling which subsetting data set down to its 1/50, I could get the subset data set (shown as above small data set) of original 150 data set. And these two have similar prior probability (here they are same). However, simple random sampling cannot guarantee. Therefore, I support stratified sampling will suit for NB modeling.

In sum, is it possible that this is just a typo? The "not" should be wiped out? Looking forward to your suggestions.

$\endgroup$
  • $\begingroup$ If the book is Predictive Analytics and Data Mining: Concepts and Practice with RapidMiner By Vijay Kotu, Bala Deshpande, then the "class" in "class-stratified sampling" is the outcome classification. So that this is sampling on the outcome, not "ordinary" stratified sampling. $\endgroup$ – Steve Samuels Sep 5 '15 at 23:28
  • $\begingroup$ @SteveSamuels Appreciate your comments, Steve. You are right about the book name. And could you expand on "sampling on the outcome, not ordinary stratified sampling"? I edited my questions, enriched the points where I felt confused, improved the formatting. It might help. $\endgroup$ – Puriney Sep 6 '15 at 3:35
  • 1
    $\begingroup$ Stratified random sampling is one method for obtaining a data set that "represents" the population. I don't know if this is what the authors meant $\endgroup$ – Steve Samuels Sep 7 '15 at 4:24
0
$\begingroup$

IMHO, I think although Naive Bayesian assumes independence between features, but in reality this is only an approximation. That means there exists correlation between features somehow. If the correlations differs in your subgroup, any subgroup's correlation end up being different from the overall population, which leads to bias. On the other hand, stratified sampling imposed exclusiveness between sample points, which destroyed independence between data points. Actually, I doubt it can be used in any inference about the population if you don't know exact parameters in subgroup.

$\endgroup$
0
$\begingroup$

I realized that my assumption might be wrong, which states "I will expect the P(Y) of subset will be close to P(Y) of the original training set."

The parameters of probability distribution (or underlying data structure) is unknown. Bayesian methods begin with a prior probability (prior degree of belief). After training model by learning more observation data, we can update the belief, i.e. post probability.

Training data set composes of features (attributes) and classes (or labels). Stratified sampling would introduce bias by the labels.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.