I'm newbie in AI

I know that Supervised Learning algorithms are divided into Classification and Regression algorithms.

But is that true of all machine learning algorithms, not just Supervised Learning? Are there any other categories than Classification and Regression?

  • 3
    $\begingroup$ There are algorithms that do both: classification and regression at the same time stats.stackexchange.com/q/245902/35989 $\endgroup$
    – Tim
    Jul 19 '20 at 10:47
  • 1
    $\begingroup$ While a bit marketingy, I found this classification of ML algorithms quite useful. Just ignore all the e-commerce applications stuff. ;) cygnismedia.com/images/post-images/… $\endgroup$
    – jhin
    Jul 19 '20 at 21:05
  • 2
    $\begingroup$ Isn't classification itself just a kind of regression? A classifier is a function of the data with a discrete range; training a classifier means fitting its parameters to best represent the training dataset. $\endgroup$ Jul 19 '20 at 21:32

All unsupervised algorithms, e.g.

  • clustering,
  • dimension reduction (PCA, t-sne, autoencoder,...),
  • missing value imputation,
  • outlier detection,
  • ...

Some of them might internally use regression or classification elements, but the algorithm itself is neither.

  • 1
    $\begingroup$ Pretty much the same mindset :) (+1) $\endgroup$
    – gunes
    Jul 19 '20 at 10:35
  • 6
    $\begingroup$ This is really about terminology. Clustering is all about putting things into classes, it just happens that ML people reserve the term classification to the supervised case. "Dimensionality reduction" may be the most convincing in the list. But the question is almost "do we always learn something either discrete or continuous?" $\endgroup$ Jul 19 '20 at 20:14
  • 1
    $\begingroup$ @MarcGlisse Of course, regression is just dimension reduction to one dimension. $\endgroup$ Jul 21 '20 at 4:16
  • 1
    $\begingroup$ @MarcGlisse "do we always learn something either discrete or continuous?" this sentence was the question I really wanted to ask. you increased my insight. really thank you :) $\endgroup$
    – Soulduck
    Jul 23 '20 at 13:08

No, it's much broader than that. You should at least read about the following:

  • 1
    $\begingroup$ You were 22 seconds faster (+1 :-)). $\endgroup$
    – Michael M
    Jul 19 '20 at 10:34

Generally speaking "supervised" learning", "classification" and "regression" are actually very different levels of meaning.

Supervised learning is a high level categorization of ML problems which defines all challenges where we have at least some solved/labeled data. This is opposed to unsupervised learning (we don't know the solution) and reinforcement learning (data and labels are generated procedurally).

Classification is specific goal of ML which you can compare to targets like prediction, outlier detection, dimension reduction, etc.

Finally regression is a specific mathematical algorithm which can help us achieve tasks and might be opposed to algorithms such as a Neural Net, Naive Bayes, etc.

A specific ML model can be described in all three terms:

  1. An unsupervised classification problem solved with a K-Means clustering algorithm

  2. A supervised prediction problem solved with a linear regression

  3. A reinforcement learning optimization problem solved with a monte carlo model.


First you need to know that machine learning algorithms are broadly divided into three categories-

  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Learning

But you should know that most production level machine learning pipelines use a combination of two or all of the three kinds of algorithms.

Supervised Learning takes advantage of already known labels, like whether an email is reported spam or not, how much rainfall has occured in the last 7 days, whether a lump in body is carcinogenic or not etc.

Where in Unsupervised Learning, the data is not labeled i.e. there are no clearly defined target variables (nature of email, amount of rainfall and nature of tumor are the target variables in the previous cases).

Reinforcement Learning algorithms are complex and advanced where the model learns from its previous predictions and correctness.

So, whenever there is a clearly defined target variable, you can apply a supervised learning algorithm. Regression and Classification fall into the supervised learning domain, and cannot be classified as unsupervised learning models.

And, there are many supervised learning algorithms which are not regression or classification, for example-


These are just some examples of the supervised learning algorithms. And these, along with regression and classification, do not fall under unsupervised learning algorithms. Some of the most common unsupervised learning algorithms are-


Here's a diagram-

                                    Machine Learning Algorithms
|                                    |                                          |
supervised learning         unsupervised learning           reinforcement learning
|                                            |
|--->Naive Bayes Classifier                  |--->Clustering
|--->Support Vector Machine                  |--->Neural Networks
|--->Decision Tree                           |--->Anomaly Detection
|--->Random Forest

These questions are better suited for the Data Science Stack Exchange site.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.