0
$\begingroup$

I'm not sure if this is the best place to ask a question like this. If not, please redirect me! Here goes:

Does anyone have experience with writing autoencoers in Keras? I'm getting pretty strange results. Here's a link to github gist with my model (it is only 30 lines of codes, half of which are comments/imports):

https://gist.github.com/reesepathak/118c5573e320c82cf99d1bec26dc6366

Basically the problem is I consistently get lower validation loss versus training loss. This is so weird! Any idea as to what might be going wrong?

Improve this question
$\endgroup$
1
  • $\begingroup$ As presently phrased this is a question about coding problems which would generally be off topic here (and you only offer a link to the code; when the link changes in any way your question isn't answerable). I only hesitate to close it because of the form of the answer (which could potentially make it on topic if you rephrase your question). Please rephrase to make it a question that relates more to a question about NN and autoencoders than details of an implementation in code. $\endgroup$
    – Glen_b
    Commented Jul 10, 2016 at 4:32

1 Answer 1

Reset to default
2
$\begingroup$

Training loss is based on mistakes on vectors where the network has already been told the correct answer. In the validation portion of the data, the network hasn't been told the right answer and will make more mistakes. If the discrepancy is large, we use the word "overfitting" to describe it. In that case, the network is basically memorizing a small number of answers instead of learning how to produce correct answers in general.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.