5
$\begingroup$

Recently I attended a seminar where they said that a machine learning (ML) model is just a mathematical equation.

Having been studying neural nets and deeper models as such I feel it is not a precise statement.

A ML model could be better said to be a black box in which there are signals to tune its parameters. Internally, it may be very hard to represent a model as an equation. Many models have lots of moving parts too but as long as they are end to end differentiable (e.g. in neural nets) or have some way to tune the components they can work.

Is there any way to give a precise definition of a ML model? Can all ML models be written as mathematical equations?

$\endgroup$
8
  • 7
    $\begingroup$ @a_statistician In this case the OP means "Machine Learning" as per his first tag. Yes, its confusing. That's why we ask people to expand abbreviations on first use. See How to ask good questions on CrossValidated under Style or Top ten reasons to close under the answer "Questions which contain a large number of undefined abbreviations or acronyms", or see one on avoiding acronyms $\endgroup$
    – Glen_b
    May 25, 2017 at 4:30
  • 2
    $\begingroup$ You kind of answer your own first question. Moreover, a Machine Learning model is an theoretical solution based on the many possible approaches offered by the field, many of which relies on Artificial Intelligence. Actually, Wiki defines it as a sub-field of Computer Science, it's a really nonsense oversimplification to assert it's just a mathematical model. $\endgroup$ May 25, 2017 at 4:34
  • 1
    $\begingroup$ There are lots of inequalities in Machine Learning too. $\endgroup$ Apr 17, 2018 at 15:28
  • 1
    $\begingroup$ Richard Feynman pointed out (somewhere in his Lectures on Physics, I believe) that all of physics--that is, the description of how everything in the universe works--can be reduced to a single mathematical equation. This suggests that the answer to your question comes down to eliminating the phrase "just a." $\endgroup$
    – whuber
    Apr 17, 2018 at 15:40
  • 1
    $\begingroup$ @MarkL.Stone The Hoeffding's inequality and the Jensen's inequality come to my mind. Also the McDiarmid's inequality (know little about it but have seen it in the papers). $\endgroup$ Apr 18, 2018 at 4:30

4 Answers 4

5
$\begingroup$

There are some interesting differences between Mathematical Models and Machine Learning models which have important consequences.

Consider, Isaac Newton's Second Law of Motion:

$ F = MA $

This simple equation (or model) provides us with much more than just an answer. Think of the computer in Hitchhikers Guide to Galaxy that answers the question

 What is the meaning of life?

with

"42". 

This illustrates the problem of being given an answer without a theoretical framework or context in which to understand it.

From the equation F=MA I can also derive:

$ M = \frac{F}{A}$

Using Calculus I an also derive:

$ F = M\ddot{X} $

since acceleration is the 2nd derivative of displacement.

Mathematical Models are created through understanding the problem. So when they work, they don't just give us an answer, they also give us some insight into the problem and it's underlying mechanisms.

One advantage of machine learning techniques is that they can give accurate approximations or answers without requiring any theory or understanding upfront. However, the answer is provided without context and can't be as easily manipulated to create further knowledge or hypotheses.

Also, a neural network can't recreate a simple equation like F=MA without using considerable computation compared to a simple multiplication. This gives you some idea why machine learning is sometimes referred to as a "black box" or "brute force" approach. See Simplest way for ANN to learn F = MA?

$\endgroup$
1
$\begingroup$

I think you are rightly skeptical of the statement:

a machine learning (ML) model is just a mathematical equation.

What I particularly dislike is the careless use of the word "equation". The word equation has a specific meaning in mathematics:

An equation is a statement of equality between (at least) two quantities or expressions.

An equation always contains the reserved symbol "$=$".

I hope you agree that this does not quite capture what a trained/estimated machine learning model is.

There is however a mathematical concept that does capture what a large class of trained machine learning models fall into:

A trained machine learning model is nothing more than a function.

To recap the definition: we call $f$ a function from set $X$ into set $Y$, written

$$ f : X \to Y, $$

if and only if each $x \in X$ is associated with a unique $y \in Y$. You could therefore view $f$ as a set of ordered pairs $(x,y)$ (with $x \in X, y \in Y$) such that each $x$ occurs exactly once.

The previous does not capture the case where an input $x$ undergoes non-deterministic transformations before being returned as $y$. In that case, it would be better to call the machine learning model an algorithm (or recipe). Or even, a machine – in the conceptual meaning of that word.


See also:

$\endgroup$
1
  • $\begingroup$ @downvoter care to comment? #Karma $\endgroup$
    – Jim
    Apr 17, 2018 at 19:54
0
$\begingroup$

Let's break down a Machine Learning Task into two parts, training and testing.

In the training phase, you specify a framework for the relationship (structure) between input an output. Perhaps you choose to use a neural network with specified layers. In the training phase, you use an algorithm to determine weights of the network paths.

What you end up with a long equation including the inputs, weights, biases, etc., which is how predictions are made.

In order to make a prediction on new values, you must have an equation, $y_{pred}=...$. If you make a prediction on the same input with the same trained model, it will always yield the same answer because it is a mathematical equation, regardless of how complicated and unintuitive it is.


To Summarize

According to Wikipedia, A neural network is a mathematical model. So we have a mathematical model (neural network) that is trained using an algorithm or heuristic to generate a mathematical equation that defines how the system works (even if we can't interpret the system) and how to make a prediction on any new value.

Determining if it is "just" a mathematical equation depends on one's interpretation of "just." Since the end result of a machine learning model is generally an equation in the form $y_{pred}=...$, I would agree that it is just a mathematical equation, in the same spirit that least squares regression is just an equation.

$\endgroup$
-1
$\begingroup$

Yes, machine learning models are mathematical models. Most machine learning models rely on a combination of linear algebra, calculus, probability theory or other math concepts to predict something from some labeled (supervised) or unlabeled (unsupervised) data.

$\endgroup$
1
  • 1
    $\begingroup$ I would say there is a difference between a Mathematical Model, and something which merely uses Maths. $\endgroup$
    – COOLBEANS
    Apr 16, 2018 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.