I'm training on a dataset with 3600 columns. 100948 training rows & 25238 validation rows.
These are the R commands I'm using:
train_hex = h2o.importFile(localH2O, paste(getwd(), "TrainingData.gz", sep="/")) train_hex_split = h2o.splitFrame(train_hex, ratios = 0.8) h2o.deeplearning(x = 242:(ncol(train_hex) - 18), y = "Predict-A", training_frame = train_hex_split[], validation_frame = train_hex_split[], activation = "RectifierWithDropout", hidden = c(1600, 1600, 1600), hidden_dropout_ratios = c(0.1, 0.1, 0.1), epochs = 100, initial_weight_distribution = "UniformAdaptive", input_dropout_ratio = 0.1, l1 = 0.00001, l2 = 0, loss = "Automatic", shuffle_training_data = TRUE)
After about 24h of training this is the result I'm seeing.
(blue: training; orange: validation)
I'm pretty new to the field and I'm struggling to understand what's going on here? I like the training progress on the blue line. But I don't even understand how the validation can start of lower than the training!?
I'd appreciate any help.