I would like to implement (L2-regularized) Logistic Regression, (L2 regularized) SVM and Random Forest for multiclass classification in Matlab (without using a toolbox or the corresponding functions in Matlab).

Does somebody know easy implementable pseudocode or easy Code (in Java, R, Python etc.) which I can use for my implementation in Matlab.


I could offer you the L2 logistic regression in Python: example & code, I added a parameter to toggle between Stochastic Gradient Descent and Gradient Descent, I am not sure which one you want.

I think I remember that Andrew Ng covered SVMs in his ML course on coursera, which used MATLAB. I just did a quick google search and found some github repos, e.g., this one which implement the code. You just need to look for the correct section or exercise.

Random Forests are a little bit tricker, since you first want to implement the decision trees efficiently. I'd probably not recommend to implement those yourself, determining the best split along continuous numerical variables can be quite costly. The only implementation that comes to my mind would be the one in scikit-learn here

  • $\begingroup$ Thank you very much for your answer. I will try to adopt your Python Code. Is SGD or GD better? Do you also know some sample code for kernelized (RBF kernel) SVMs? So I think I will not do Random Forest by my own. But what about e.g. Deep Neural Networks, AdaBoost, Gaussian Naive Bayes (all for classification). Is this also difficult for self-implementation and again do you know some good pseudocode / code for it? $\endgroup$ – machinery May 2 '15 at 20:19
  • $\begingroup$ Hi, Thomas, it depends on the situation or context whether SGD or GD is better. Those are just the simplest examples of optimization algorithm and very, very easy to implement. I have written some sections about the theory behind it in context of adaptive linear neurons here if it helps to clarify the difference. $\endgroup$ – user39663 May 3 '15 at 18:00
  • $\begingroup$ Also here, I would advice against an own implementation for performance reasons -- I would rather write a wrapper for LIBSVM to do kernelized SVMs. I don't know any easy implementation for kernel SVM, but I implemented kernel PCA in simple Python code here, the concept (kernel trick) is essentially the same. $\endgroup$ – user39663 May 3 '15 at 18:03
  • $\begingroup$ I don't know if it makes sense to implement Deep neural nets in matlab for performance reason. If you prefer Python as alternative, there is a great library to do that called theano. You can find a lot of tutorials and examples on their website $\endgroup$ – user39663 May 3 '15 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.