1
$\begingroup$

I am trying build a Naive Bayes classifier from data pulled from scientific papers. I want to use the reported variable distribution parameters to approximate a dataset which I can use to train the Naive Bayes classifier (since usually there is no access to raw data).

For example, a paper reports the distribution of the height for apple trees and orange trees, as well as whether the tree is in the north or south side of the orchard. These distributions/categories are reported with respect to the type of tree. I also know what percentage of trees are apple, and what percentage are orange (prior probability). However, the raw data is not available. I am trying to convert such information into the training data for a Naive Bayes classifier (in this example, I would want to end up with the probability that a tree is an apple tree or an orange tree).

Can I build an approximation of the multivariate dataset from only the distribution parameters (without knowledge of covariance between parameters or raw data)?

OR can I more directly approximate the P(Xi | A) for a Naive Bayes classifier from the distribution of Xi without reconstructing an estimation of the dataset?

$\endgroup$
1
$\begingroup$

Is it the type of tree that you want to be predicting? Just to be sure you have the data I think you have, I'm going to paraphrase. You have a histogram for each attribute given the type of tree, is that right? If that's correct...

To apply Naive Bayes, you'll need an approximation for $P(X_i|C_k)$. To get $P(X_i|C_k)$, you could fit some family of probability distributions to each histogram (e.g. if the an attribute is normally distributed, you could fit a Gaussian to it). If the histogram is hard to approximate via a known probability distribution, you could use a generalized linear model or a mixture model. Then you can predict the probability of a type of tree $C_k$ via Naive Bayes like this:

$P(C_k|x_1, x_2, \dots, x_d) = \frac{P(C_k)\prod_1^dP(x_i|C_k)}{\sum_1^kP(C_k)\prod_1^dP(x_i|C_k)}$

I you don't actually need this to be a probability (you only care about proper classification), there's no need to normalize via the denominator. Leaving the priors out for unbalanced data is also optional.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.