Return to Question

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with N targets (analytical $\ddot{q}$, so how many doublets $q$, $\dot{q}$ i'm feeding to the MLP) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting $\ddot{q}$, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.

For the training part, I'm feeding the net analytical trajectories where points are shuffled, with the following parallel in mind:
training on batches of a continuous part of a curve, then going through the next part and so on, is like training an image recognition net by feeding ALL cats first, then ALL dogs etc, slowing down the learning of different features by focusing on the ones of a specific class. Again, this is (i think) true because of the pointwise nature of the training.
Here x = ($q$, $\dot{q}$), xt = ($\dot{q}$, $\ddot{q}$) (it's easier to pair them like this for how the code is built)

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with N targets (analytical $\ddot{q}$, so how many doublets $q$, $\dot{q}$ i'm feeding to the MLP) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting $\ddot{q}$, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.

For the training part, I'm feeding the net analytical trajectories where points are shuffled, with the following parallel in mind:
training on batches of a continuous part of a curve, then going through the next part and so on, is like training an image recognition net by feeding ALL cats first, then ALL dogs etc, slowing down the learning of different features by focusing on the ones of a specific class. Again, this is (i think) true because of the pointwise nature of the training.
Here x = ($q$, $\dot{q}$), xt = ($\dot{q}$, $\ddot{q}$) (it's easier to pair them like this for how the code is built)

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with N targets (analytical $\ddot{q}$, so how many doublets $q$, $\dot{q}$ i'm feeding to the MLP) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting qtt, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with N targets (analytical $\ddot{q}$, so how many doublets $q$, $\dot{q}$ i'm feeding to the MLP) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting $\ddot{q}$, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with my targets (analytical $\ddot{q}$) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting qtt, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.

Because I'm trying to fit a scalar function, the MLP can be thought of as an equation in P parameters, where P is the number of trainable params of the net.
Training it and comparing predictions with N targets (analytical $\ddot{q}$, so how many doublets $q$, $\dot{q}$ i'm feeding to the MLP) imposes N restraints, so I can think of it as solving a system of N equations in P parameters.
As such, my intuition tells me I need to have AT LEAST N=P points ($q$, $\dot{q}$) to correctly solve it.
In the following example, I have P = 3361. Feeding it N = 4096 points will be enough to train them. Training with N = 16384 points should be better on one hand (because a larger training set can better generalise the predictions, given some initial conditions), but following the N eq in P variables reasoning, it feels like overdetermining the system. I have tried various configurations of this, and having N >> P seems better at fitting qtt, though never providing correct trajectiories because of the aformentioned propagation of errors in integration.