Optimizing parameters of fully connected layers I have a conceptual problem choosing the best strategy for training a neural network. So, let me explain my situation. I have trained an autoencoder on a huge (unlabeled) dataset using train and validation split. Now I would like to extend the architure by fully connected layers to predict a small labelled dataset similar to the approach described here. As input features to the fully connected layers, the encoding part of the autoencoder will be used.
The parameters of the fully connected layers must be optimized. For SVM and Random Forest I would use nested cross-validation to optimize the hyparameters using random or grid search (or just one cross-validation with default parameters). With fully connected layers this is not a good options I think. Usually, I would split the data into train, test and validation set but my dataset consists only of 800 points in class 1 and 500 points in class 2. So train, test, validation split is probably also not a good option.
Using nested-crossvalidation and random search is very time consuming for fully connected layers. Moreover, using cross-validation I cannot observe the training and validation loss anymore and adjust the parameters acoordingly (as I did for tuning the autonecoder). So cross-validation does not work for manual tuning the network.
What is the best strategy to optimize the fully connected layers in my case?
 A: tl;dr
Since you have a small dataset, I'd go with cross-validation
Reasoning
Usually, small datasets don't come hand in hand with slow computation times. If your dataset is small you can typically use cross-validation to optimize hyperparameters. If it is very large and training times are very slow then you can safely use a train/val/test split.  
In particular

Using nested-crossvalidation and random search is very time consuming for fully connected layers.

I disagree with this point. It shouldn't be slow in such a small dataset. I don't think that it is slower let's say than training a SVM.
One reason might be that your model is larger than it has to be (i.e. lots of parameters). If your dataset is small then you shouldn't use very large models because they will be harder to train and will probably be more prone to overfitting.
Another issue might be that your choice of hyperparameters is slowing down the process. Possibly a low learning rate, or maybe too many epochs.

Moreover, using cross-validation I cannot observe the training and validation loss anymore and adjust the parameters acoordingly (as I did for tuning the autonecoder)

I don't get this point either. I'm assuming you refer to things you do during training (e.g. learning rate adjustments). During $k$-fold CV you train your model $k$ times, each is a regular training process. Whatever you do with a train and a validation set you can do in each of the CV folds. 
