I'm looking into understanding the Deeplearning anomaly detection algorithm provided by h2o. I tried to recreate an example below. Perhaps some of these questions are basic, but I'm trying to better understand this technique as I'm not too familiar with the approach. There seems to be a wealth of information around 'deep learning' but it often is overly technical or theoretical , making it hard for me to connect the dots if you will....
That said... how can I properly 'estimate' an ideal number of hidden layers in an h2o Autoencoders model? How can I tune this model properly?
- My Understanding is generally layers is most important parameter to tune. Are there good rules of thumb on how to choose this value? A given value such as 5? a % of features? I know it probably 'depends', but at same time I don't want to spend hours and hours grid-searching when it is possible that a 'rule-of-thumb' value(s) are more often than not close enough.
How can I understand the print-outs of an h2o Autoencoders model output? The values returned seem to be:
- layer
- units
- weight_rms
- mean_bias
- etc...
How should I be looking at this to ensure that my un-supervised model is appropriate?
- This is probably answered by the above questions but...looking at models with a variety of different hidden layers you get some pretty different looking reconstruction MSE plots. How do I know what one is ideal?
This likely isn't the best dataset to show this problem since it's relatively small, but here is an example:
library(mlbench)
library(h2o)
library(tidyverse)
data('Sonar')
h2o.init()
df=Sonar
h2odf = as.h2o(df)
features = setdiff(names(df), 'Class')
hyperparms = list(hidden=c(1,5,10,15,20,25,35))
grid=h2o.grid(algorithm = 'deeplearning', x = features, training_frame = h2odf, epochs=100, autoencoder=TRUE, hyper_params =hyperparms , activation='Tanh', grid_id='understanding_dl')
check_models=c(as.data.frame(grid@summary_table)$model_ids)
initialize_df =h2o::h2o.createFrame(rows=nrow(df), cols = 7)
counter = 0
for (each_model in check_models){
print(h2o.getModel(each_model))
counter = counter+1
initialize_df[,counter]=h2o.anomaly(h2o.getModel(each_model), data=h2odf)
}
#interesting to see correlation of values
cor(initialize_df)
recon_mse=as.data.frame(initialize_df)
names(recon_mse)=c(hyperparms$hidden)
recon_mse_gathered = recon_mse%>% mutate(record=seq(1,nrow(df)))
df2 <- melt(recon_mse_gathered , id.vars = 'record', variable.name = 'hidden')
# density of recon MSE w/ each hidden layer
df2 %>% ggplot(aes(x=value, color=hidden))+geom_density()+labs(x='Recon MSE')