3
$\begingroup$

I am looking for an online learning classifier that is highly adaptable and has only short-term memory. I need such a think in a object tracking system with high-dimensional feature vectors.

Maybe a system with a constant learning rate could be useful?

Any pointers to literature? Any useful advice?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

You Should look up stochastic gradient descent. See http://scikit-learn.org/stable/ and wikipedia.

I also think Kalman Filter will do what you ask for. Just set a high learning rate.

However to high learning rate removes the "learning" from the system. If past experience have no effect (or negible) on the decision then no learning.

Improve this answer
$\endgroup$
9
  • $\begingroup$ I am not sure the Kalman Filter would work in this situation: I am going to have high-dimensional feature vectors (colors, edge orientations...), while I saw the Kalman filter working just on coordinates and a few hidden variables. Moreover the Kalman filter, in the end is going to be just a single learned multivariate normal distribution, while maybe I need something more complex. But I will have a look on stochastic gradient descent, this might be helpful. $\endgroup$
    – fstab
    Jul 28, 2015 at 9:24
  • $\begingroup$ By the way, how would you use the stochastic gradient descent? How would you use if for a scoring/classification system such the one I need? $\endgroup$
    – fstab
    Jul 28, 2015 at 9:32
  • $\begingroup$ Can you tell us a little more about the underlying model? $\endgroup$
    – NegativeFeedbackLoop
    Jul 28, 2015 at 14:11
  • $\begingroup$ It's a football player tracker, and the classifier should learn from the rectangle containing a specific player, over several frames $\endgroup$
    – fstab
    Jul 28, 2015 at 15:51
  • $\begingroup$ As far as i know SGD assumes a linear classifier. Seems like you maybe do something more complex. What kinda classifier do you use or think you want to use? $\endgroup$
    – NegativeFeedbackLoop
    Jul 30, 2015 at 7:53
1
$\begingroup$

I worked on a kernel-based algorithm that allows you to explicitly deal with forgetting. It's called kernel recursive least-squares tracker (journal paper link) Data can be anything as long as you can formulate a kernel on them.

The algorithm is formulated for regression though, but one of the cited papers formulates the same framework for classification (though only for stationary scenarios): L. Csató and M. Opper. Sparse online Gaussian processes. Neural Computation, 14(2):641-669, 2002.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.