3
Triplet models are notoriously tricky to train. Before starting a triplet loss project, I strongly recommend reading "FaceNet: A Unified Embedding for Face Recognition and Clustering" by Florian Schroff, Dmitry Kalenichenko, James Philbin because it outlines some of the key problems that arise when using triplet losses, as well as suggested ...
3
I found two recent works which seem relevant --
Triple descent and the two kinds of overfitting: Where & why do they appear?
The claim is that there are two (sample-wise) peaks: one when number of inputs N equals the input dimension D, and one when N equals the number of parameters P. For linear models, D=P, so only one peak is observed. For highly non-...
2
You should not use augmented data in the validation nor in the test set.
Validation and test set are purely used for hyperparameter tuning and estimating the final performance, i.e. estimating the generalization error. These two data sets should be as close as possible to other data, which you could have acquired, but you actually haven not, i.e. your true ...
1
Consider the simpler case, where you have one target, $y$. The input-output relationship in neural networks is, in general,
$$y=f(\theta, x)+\epsilon$$
where, $y$ is the target, $x$ is the feature vector, $\theta$ is the set of parameters, and $\epsilon$ is the random error. It's typical to assume that the random error is distributed normally with zero-mean ...
1
It appears this is due to the instability of the hidden features and the dynamics of gradient descent with these shifting features.
I did some experiments fitting with a small network with 3 hidden neurons, fitting one dimensional nonlinear data.
If you freeze the first layer, and only run gradient descent on the last layer, then the residuals go to zero:
...
1
The permutation importance test is to shuffle the feature values. It's not to shuffle the labels. The labels stay fixed all the time. It's to see how much score changed by shuffling the feature values. If the feature is irrelevant, the shuffling will not affect much on the score. But if the feature is important for the model, the shuffling of the feature ...
1
Plotting error/accuracy against number of epochs is a way to see how fast is your neural network approaching a minima. Usually, one would observe a low accuracy for a small number of epochs. With an increasing number of epochs, accuracy should increase. Hence, your plot is rather weird! Are you sure it's correct?
But now what will be the overall performance(...
1
I see two issues.
Accuracy is a surprisingly bad performance metric. If you evaluate your model using a so-called proper scoring rule like cross-entropy loss or Brier score, you may find that the out-of-sample performance improves even though accuracy decreases. This is because accuracy relies on a threshold, and the threshold that gives the beat accuracy ...
Only top voted, non community-wiki answers of a minimum length are eligible
