Recently, I stared working on a machine learning competition hosted on Kagge.com.

As the first step, a quick and dirty system was developed using Logistic Regression (LR). After running the system with increasing the number of training examples step by step and plotting the learning curve, I realized that my system is suffering from the high-biased issue. In order to overcome this issue, I increased the number of features step by step and measure the training and cross validation errors. Unfortunately, this did not show a significant improvement and both training and cross validation accuracies remained in the 76% - 79% range.

At this time I’m considering to follow one of the following avenues.

  1. Try few more learning algorithms (single layer NN, SVM or decision tree) and take the majority vote.
  2. Since this is a text (actually web pages) analysis problem, instead of unigram, try bigram (and trigram) with LR and check whether I can achieve some significant improvement.

Your expertise advices are highly appreciated.


1 Answer 1


I'm working on that competition (stumbleUpon) as well. Although I'm not an expert in machine learning, I would say that maybe the most important part of such an analysis is the so-called 'feature engineering'. That is, do not use all the HTML code as a text, I'm not sure that much information is hidden in the text itself (because you feed your algorithm with a lot of junk). Instead, try to identify more specific features that you could extract from the HTML code such as 'Number of tags' etc.

You could also use the AlchemyAPI (it's free) to complete some missing data. I'd also advise to visit some of the webpages yourself and identify anything that shows whether each page is 'evergreen' or not. Then you can use these hints and extract the relevant data from the HTML code.

Of course, it's very easy to try out various algorithms (trees in all their forms, nnets and kernel SVMs) once you have pre-processed your data.

  • 1
    $\begingroup$ @upul, don't reinvent the wheel (unigram ec), use alchemy's api - keyword extraction etc $\endgroup$
    – seanv507
    Commented Aug 21, 2013 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.