For a university project, we want to use reinforcement learning from raw camera input to teach a robot to hit a ball. It works when we detect the ball and feed that to the algorithm, so raw data is the next step.
As this data is too high dimensional, we want to apply dimensionality reduction. In a first evaluation, we used PCA and other non-neural methods to reduce from 28x32 to around 20 dimensions while still be able to learn from it with a bit loss in precision. Currently, we want to see whether autoencoders also work for that.
The code base of the team was matlab, therefore we used in the first part matlab autoencoders. We used a dataset of 100 pictures and reduced to 200 dimensions. Options are mostly default, from what I remember it where up to 200 episodes.
Right now, the lab is transitioning to Python instead of Matlab. Therefore, we used Keras to build a custom autoencoder, closely following this blog entry. But when we run our autoencoder on the same dataset to same dimensions, the following happens:
The ball disappears. We have created a bigger dataset with up to 2500 of these pictures, but the ball always disappears:
Also, we used the sparse autoencoder and deep autoencoder, played with the activations and loss function, but always the same result. Has anybody tips on how to improve the learning so we keep the ball? Why does it work on the small dataset in Matlab, but not in Keras on the same?
Edit: I posted the code and the old and new data set on my github. I tried to subtract the mean, add Dropout, change Relu in hidden layers to sigmoids like Matlab, but I always get the removed ball.