Recently, I stared working on a machine learning competition hosted on Kagge.com.
As the first step, a quick and dirty system was developed using Logistic Regression (LR). After running the system with increasing the number of training examples step by step and plotting the learning curve, I realized that my system is suffering from the high-biased issue. In order to overcome this issue, I increased the number of features step by step and measure the training and cross validation errors. Unfortunately, this did not show a significant improvement and both training and cross validation accuracies remained in the 76% - 79% range.
At this time I’m considering to follow one of the following avenues.
- Try few more learning algorithms (single layer NN, SVM or decision tree) and take the majority vote.
- Since this is a text (actually web pages) analysis problem, instead of unigram, try bigram (and trigram) with LR and check whether I can achieve some significant improvement.
Your expertise advices are highly appreciated.