Is a machine learning model just a mathematical equation?

Question

Recently I attended a seminar where they said that a machine learning (ML) model is just a mathematical equation.

Having been studying neural nets and deeper models as such I feel it is not a precise statement.

A ML model could be better said to be a black box in which there are signals to tune its parameters. Internally, it may be very hard to represent a model as an equation. Many models have lots of moving parts too but as long as they are end to end differentiable (e.g. in neural nets) or have some way to tune the components they can work.

Is there any way to give a precise definition of a ML model? Can all ML models be written as mathematical equations?

Answer 1

There are some interesting differences between Mathematical Models and Machine Learning models which have important consequences.

Consider, Isaac Newton's Second Law of Motion:

$ F = MA $

This simple equation (or model) provides us with much more than just an answer. Think of the computer in Hitchhikers Guide to Galaxy that answers the question

 What is the meaning of life?

with

"42". 

This illustrates the problem of being given an answer without a theoretical framework or context in which to understand it.

From the equation F=MA I can also derive:

$ M = \frac{F}{A}$

Using Calculus I an also derive:

$ F = M\ddot{X} $

since acceleration is the 2nd derivative of displacement.

Mathematical Models are created through understanding the problem. So when they work, they don't just give us an answer, they also give us some insight into the problem and it's underlying mechanisms.

One advantage of machine learning techniques is that they can give accurate approximations or answers without requiring any theory or understanding upfront. However, the answer is provided without context and can't be as easily manipulated to create further knowledge or hypotheses.

Also, a neural network can't recreate a simple equation like F=MA without using considerable computation compared to a simple multiplication. This gives you some idea why machine learning is sometimes referred to as a "black box" or "brute force" approach. See Simplest way for ANN to learn F = MA?

Answer 2

I think you are rightly skeptical of the statement:

a machine learning (ML) model is just a mathematical equation.

What I particularly dislike is the careless use of the word "equation". The word equation has a specific meaning in mathematics:

An equation is a statement of equality between (at least) two quantities or expressions.

An equation always contains the reserved symbol "$=$".

I hope you agree that this does not quite capture what a trained/estimated machine learning model is.

There is however a mathematical concept that does capture what a large class of trained machine learning models fall into:

A trained machine learning model is nothing more than a function.

To recap the definition: we call $f$ a function from set $X$ into set $Y$, written

$$ f : X \to Y, $$

if and only if each $x \in X$ is associated with a unique $y \in Y$. You could therefore view $f$ as a set of ordered pairs $(x,y)$ (with $x \in X, y \in Y$) such that each $x$ occurs exactly once.

The previous does not capture the case where an input $x$ undergoes non-deterministic transformations before being returned as $y$. In that case, it would be better to call the machine learning model an algorithm (or recipe). Or even, a machine – in the conceptual meaning of that word.

---

See also:

• Wikipedia: Equation.

• Wolfram MathWorld: Equation:

  • "An equation is a mathematical expression stating that two or more quantities are the same as one another, also called an equality, formula, or identity."

    Personally, I would not use "formula" as equivalent for "equation".

• Wolfram MathWorld: Function.

Answer 3

Let's break down a Machine Learning Task into two parts, training and testing.

In the training phase, you specify a framework for the relationship (structure) between input an output. Perhaps you choose to use a neural network with specified layers. In the training phase, you use an algorithm to determine weights of the network paths.

What you end up with a long equation including the inputs, weights, biases, etc., which is how predictions are made.

In order to make a prediction on new values, you must have an equation, $y_{pred}=...$. If you make a prediction on the same input with the same trained model, it will always yield the same answer because it is a mathematical equation, regardless of how complicated and unintuitive it is.

---

To Summarize

According to Wikipedia, A neural network is a mathematical model. So we have a mathematical model (neural network) that is trained using an algorithm or heuristic to generate a mathematical equation that defines how the system works (even if we can't interpret the system) and how to make a prediction on any new value.

Determining if it is "just" a mathematical equation depends on one's interpretation of "just." Since the end result of a machine learning model is generally an equation in the form $y_{pred}=...$, I would agree that it is just a mathematical equation, in the same spirit that least squares regression is just an equation.

Answer 4

Yes, machine learning models are mathematical models. Most machine learning models rely on a combination of linear algebra, calculus, probability theory or other math concepts to predict something from some labeled (supervised) or unlabeled (unsupervised) data.