2
$\begingroup$

I want to build classifier using naive bayes. I know that naive bayes is supervised learning that requires labeled data for training set. However, I only have data with no label on it. Is there any method to give label to unlabeled training data or we only need to label it manually? I want to label my data with two classes. My data is all in continuous variable and has three attributes to determine the class.

Edit:

I've found useful java program which uses MVE for multivariate outlier detection in this link : http://www.kimvdlinde.com/professional/mve.html

I use that program to label the data which is all in continuous variable. What kind of method that I can use to label data if my data has three categorical attribute? Is it still possible to label it manually?

Improve this question
$\endgroup$
4
  • $\begingroup$ How many classes do you think you have? If the answer is two, certain methods become an option. Can you also give a better idea about the sort of data you have (is it documents or something with all continuous variables etc)? Maybe update your question with these details? $\endgroup$ – Deathkill14 May 28 '14 at 7:40
  • $\begingroup$ I have two classes. My data is all in continuous variable. What kind of method that I should take? $\endgroup$ – wakawaka May 28 '14 at 9:12
  • $\begingroup$ And how many dimensions is it? Is it just univariate? $\endgroup$ – Deathkill14 May 28 '14 at 9:14
  • $\begingroup$ It has three dimensions. What kind of method that I should take? $\endgroup$ – wakawaka May 28 '14 at 9:25
1
$\begingroup$

If the data was only one or two dimensions it would be fairly easy to assign class labels from visual inspection. However, in three or more dimensions, more advanced methods are required. One way to look at your problem is as one of outlier detection. If one of your classes has more than 50% of the observations, then you can consider this the majority group and the second class as outliers. To help crystallize that concept, I will give a definition:

An outlier is an observation which does not come from the distribution of the majority of the data.

With this in mind, you can try to use an outlier detection method. These come from the field of robust statistics, and tend to have some nice statistical properties that some machine learning algorithms lack. They tend to be very good at detection a majority group because they were specifically designed for that.

A number of these are implemented in R. Since your data is continuous, options include the MCD, SDE and MVE, implemented in rrcov package as CovMcd, CovMved, and CovSde respectively. Another option is FastPCS, which is implemented in the FastPCS package. The paper on FastPCS might be a good introduction to how these methods work. Namely, you might want to check it out to learn a bit more about the concept of $\alpha$ that these methods use. It is the percentage of the dataset you expect to belong to the majority class. To be safe, you can always set $\alpha$ to 0.5 and plot the distances of the observations (all of the algorithms should have some distance output for the observations after they do reweighting). If you see a big jump in the distances between two groups in the data, and some observations from the first clump that are still considered outliers, you can probably increase $\alpha$ a bit to add those in - it may be that $\alpha =0.5$ is a bit too conservative.

Make sure to follow examples on the implementation of the methods in their packages to run them correctly.

In the end, these methods will identify the "good" data and "outliers," and you can give these two groups your class labels.

Improve this answer
$\endgroup$
4
  • $\begingroup$ What if my data has discrete variable? Is it still possible to use multivariate outlier detection? $\endgroup$ – wakawaka May 28 '14 at 12:23
  • $\begingroup$ Typically, this can cause problems for these methods because they assume that each observation is at a unique location in the 3-d space (the assumption of general position). You could try a robust PCA approach and set the number of dimensions you want to use to 2. That will probably make the data continuous enough to work with. Going down this path will lead to increasing complication though. $\endgroup$ – Deathkill14 May 28 '14 at 12:27
  • $\begingroup$ @Zhubarb, that is true, but in the comments to the question wakawaka says he only has two classes. $\endgroup$ – Deathkill14 May 28 '14 at 12:34
  • $\begingroup$ If I have three categorical attribute in my data, what should I do to label the data (two classes data)? I still don't understand how to do that in categorical attribute. Sorry if I am asking to much, I am newbie in data mining. $\endgroup$ – wakawaka May 28 '14 at 14:37
1
$\begingroup$

What you need is a good semi-supervised training method for your data. If you need, take a look: http://en.wikipedia.org/wiki/Semi-supervised_learning

A method that is used with Naive-Bayes abd other algorithms is PU-learning where you need only a fraction of the data of one class to be labeled. You can take a look at this for more information: https://www.cs.iastate.edu/~yetianc/cs573/files/CS573_ProjectReport_YetianChen.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.