1
$\begingroup$

I have been developing a simple autoencoder model using PyTorch by which I am training the reconstructed output to be the same and input and also do regression on the hidden layer to predict a single integer value, at the same time. I have been using 5 fold cross-validation method to evaluate the quality of the training process. As a result, I am obtaining a high RMSE and equally high MAE scores in both the training and validation phase.

Could anybody guide me on how to further diagnose this problem? I have checked my model architecture for its correctness, and it is valid.

Improve this question
$\endgroup$
5
  • $\begingroup$ assuming correctness, it seems like underfitting. $\endgroup$
    – gunes
    Oct 17, 2021 at 15:43
  • $\begingroup$ Yeah, that's the result I am getting actually. The output values are similar to each other, with the only variation happening in units digit and the decimals. Could you suggest me on how to further avoid underfitting? $\endgroup$
    – Sahil Yerawar
    Oct 18, 2021 at 11:33
  • $\begingroup$ Did you try increasing the complexity or reducing the amount of regularization if you’re applying? $\endgroup$
    – gunes
    Oct 18, 2021 at 11:53
  • $\begingroup$ For regularization, there is no batch norm applied on any layer, nor there is any regularization constant used in loss function. The loss function consists of reconstruction loss(rmse) and regression prediction loss (rmse). The model complexity is fairly simple in terms of number of layers and number of nodes in each layer. Increasing complexity is a thing to think about, since the generated data vectors deteriorate in quality with increasing vector size. $\endgroup$
    – Sahil Yerawar
    Oct 18, 2021 at 13:39
  • $\begingroup$ @gunes do you want the model architecture to solve it further? $\endgroup$
    – Sahil Yerawar
    Oct 25, 2021 at 6:40

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.