What kind of impact do autoencoders have on final model performance when compared to models trained only on supervised data? For example, say we have two datasets, a labeled set (I will call it df_labeled) of nrows=200k and an unlabeled dataset (df_unlabeled) of nrows=800k and we want to build a binary classifier. I clearly, have the option of building my model only using df_labeled but I want to take advantage of having df_unlabeled since the data is gathered from closely related sources. 
To give further context to my problem, I am working with spending data for consumers on different products which are binned broadly into different categories like Food, Travel, Education, Health, etc. At present I am training on data from a specific category to predict on future behavior within that category. But as you can imagine consumers are not constrained to one particular category and make purchase across category, so there might be additional correlation to learn if the complete data set is taken into account. I want to make a generic model using the complete dataset to predict on future behavior of consumers in a specific category and without having to compromise a lot on performance. 
I am thinking of using an autoencoder to learn the correlations in the complete dataset (without the category tag) and then run a further training step using h2o.deeplearning on data labeled with specific categories.
Do models built using this approach perform better than models just trained on df_labeled data ? I am looking for some advice on this approach before embarking on it full throttle.
 A: In general, one would hope that an autoencoder will create a very useful low dimensional representation of data (an embedding) that summarizes the data in a very usuable way without losing important details. If it does that really well, then it is often much more efficient to build models on small supervised tasks with the embeddings as your input (see e.g. Word2vec).
However, an autoencoder may learn to encode a lot of information irrelevant to your task at hand and may discard subtle information that is crucial for your task (but where omitting it does not get penalized enough by the loss function used to construct the autoencoder - at least relative to how much unlabeled data you have). 
This is particularly of concern, if the unlabeled data is of limited size so that you can realistically only create a relatively small dimensional embedding space (insufficient unlabeled training data to create something more complex). This is less of a concern, if you have an enormous amount of unlabeled data and much, much less labeled data (e.g. for NLP every Twitter message ever vs. a few hundred or thousand tickets posted in a customer support portal). In that case, you can create a sufficiently complex embedding that captures so much subtle detail that it becomes a lot less likely that you loose anything important for your task of interest by using the embedding to represent your data. The other concern is, that the usefulness of an embedding it also depends on whether what is "interesting" in your labelled data is at all present in your unlabeled data. If the unlabeled data is from a similar data source, then this reduces the concern, while if the types of data are just completely different (e.g. using an autoencoder to create an embedding for temperature patterns in your living room and then running the brightness of the sun through the autoencoder to get embeddings to predict sunflares is probably a bad idea), the gap between what the autoencoder encodes into an embedding and what is really going on in your data may be too huge for this to be useful.
In your case, I'd speculate that it is unclear and needs to be tried. It may depend on whether the two datasets are very similar and whether your task of interest relies a lot on "obvious" features needed for an autoencoder. Thus, you probably have to just try it both ways and compare the performance on a validation set not used for training. 
