11
$\begingroup$

I'm newbie in AI

I know that Supervised Learning algorithms are divided into Classification and Regression algorithms.

But is that true of all machine learning algorithms, not just Supervised Learning? Are there any other categories than Classification and Regression?

$\endgroup$
3
  • 3
    $\begingroup$ There are algorithms that do both: classification and regression at the same time stats.stackexchange.com/q/245902/35989 $\endgroup$
    – Tim
    Commented Jul 19, 2020 at 10:47
  • 1
    $\begingroup$ While a bit marketingy, I found this classification of ML algorithms quite useful. Just ignore all the e-commerce applications stuff. ;) cygnismedia.com/images/post-images/… $\endgroup$
    – Eike P.
    Commented Jul 19, 2020 at 21:05
  • 2
    $\begingroup$ Isn't classification itself just a kind of regression? A classifier is a function of the data with a discrete range; training a classifier means fitting its parameters to best represent the training dataset. $\endgroup$ Commented Jul 19, 2020 at 21:32

4 Answers 4

13
$\begingroup$

All unsupervised algorithms, e.g.

  • clustering,
  • dimension reduction (PCA, t-sne, autoencoder,...),
  • missing value imputation,
  • outlier detection,
  • ...

Some of them might internally use regression or classification elements, but the algorithm itself is neither.

$\endgroup$
4
  • 1
    $\begingroup$ Pretty much the same mindset :) (+1) $\endgroup$
    – gunes
    Commented Jul 19, 2020 at 10:35
  • 6
    $\begingroup$ This is really about terminology. Clustering is all about putting things into classes, it just happens that ML people reserve the term classification to the supervised case. "Dimensionality reduction" may be the most convincing in the list. But the question is almost "do we always learn something either discrete or continuous?" $\endgroup$ Commented Jul 19, 2020 at 20:14
  • 1
    $\begingroup$ @MarcGlisse Of course, regression is just dimension reduction to one dimension. $\endgroup$ Commented Jul 21, 2020 at 4:16
  • 1
    $\begingroup$ @MarcGlisse "do we always learn something either discrete or continuous?" this sentence was the question I really wanted to ask. you increased my insight. really thank you :) $\endgroup$
    – Soulduck
    Commented Jul 23, 2020 at 13:08
11
$\begingroup$

No, it's much broader than that. You should at least read about the following:

$\endgroup$
1
  • 1
    $\begingroup$ You were 22 seconds faster (+1 :-)). $\endgroup$
    – Michael M
    Commented Jul 19, 2020 at 10:34
1
$\begingroup$

Generally speaking "supervised" learning", "classification" and "regression" are actually very different levels of meaning.

Supervised learning is a high level categorization of ML problems which defines all challenges where we have at least some solved/labeled data. This is opposed to unsupervised learning (we don't know the solution) and reinforcement learning (data and labels are generated procedurally).

Classification is specific goal of ML which you can compare to targets like prediction, outlier detection, dimension reduction, etc.

Finally regression is a specific mathematical algorithm which can help us achieve tasks and might be opposed to algorithms such as a Neural Net, Naive Bayes, etc.

A specific ML model can be described in all three terms:

  1. An unsupervised classification problem solved with a K-Means clustering algorithm

  2. A supervised prediction problem solved with a linear regression

  3. A reinforcement learning optimization problem solved with a monte carlo model.

$\endgroup$
0
$\begingroup$

First you need to know that machine learning algorithms are broadly divided into three categories-

  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Learning

But you should know that most production level machine learning pipelines use a combination of two or all of the three kinds of algorithms.

Supervised Learning takes advantage of already known labels, like whether an email is reported spam or not, how much rainfall has occured in the last 7 days, whether a lump in body is carcinogenic or not etc.

Where in Unsupervised Learning, the data is not labeled i.e. there are no clearly defined target variables (nature of email, amount of rainfall and nature of tumor are the target variables in the previous cases).

Reinforcement Learning algorithms are complex and advanced where the model learns from its previous predictions and correctness.

So, whenever there is a clearly defined target variable, you can apply a supervised learning algorithm. Regression and Classification fall into the supervised learning domain, and cannot be classified as unsupervised learning models.

And, there are many supervised learning algorithms which are not regression or classification, for example-

etc.

These are just some examples of the supervised learning algorithms. And these, along with regression and classification, do not fall under unsupervised learning algorithms. Some of the most common unsupervised learning algorithms are-

etc.

Here's a diagram-

                                    Machine Learning Algorithms
                                              |
                                              |
---------------------------------------------------------------------------------
|                                    |                                          |
supervised learning         unsupervised learning           reinforcement learning
|                                            |
|--->Naive Bayes Classifier                  |--->Clustering
|--->Support Vector Machine                  |--->Neural Networks
|--->Decision Tree                           |--->Anomaly Detection
|--->Random Forest
|--->Regression
|--->Classification

These questions are better suited for the Data Science Stack Exchange site.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.