I am combining multiple base classifiers for an ensemble classifier. Different sensors, such as an accelerometer, gyroscope and altimeter are classified individually, and their outputs are then fed into an ensemble classifier.

For accelerometer, 12 features are extracted from specific time windows, and Random Forest is used. For the altimeter, only one feature is extracted, so I am wondering which algorithm would be best for this?

I know that Random Forest works better when there are many features so I was thinking of using Naive Bayes, or Logistic Regression, but I cannot find any relevant literature to back this up?


Gradient boosting still works well when there are few features, and it excels at finding non-linearities in the data. Logistic regression is only able to find nonlinearities when you manually add polynomial terms, and this manual step is not necessary with gradient boosting. In any case, try out multiple algorithms: gradient boosting, random forest, and logistic regression, and see which performs best on cross validation or a holdout set. Literature can only offer guidelines, not guarantees, so you can only truly know by testing yourself.

  • $\begingroup$ Thanks Ryan, would naive bayes be considered a good algorithm for very few features? $\endgroup$ – other15 Jul 12 '16 at 11:07
  • $\begingroup$ Yeah, that should work $\endgroup$ – Ryan Zotti Jul 12 '16 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.