I'm having trouble setting network hidden layer size
I have dataset size [40,000,000 x 60] which has so many instances
and I tried to do regression with 4 hidden layer with size 300 (all same)
someone told me my network not working well could be problem of setting size too small and I should increase capacity of neural network
1. how should I set hidden layer size? it seems too arbitrary and I wonder if there is rule of thumb
and I used all same size in all 4 layers and there can be improvement in accuracy when setting different layer sizes and 2. I wonder if there is rule of thumb when setting layer sizes. I'm dealing with regression problem and people say highest hidden layer should be large in regression problem and wonder why.
It's quite complicated but trying to estimate gene relevance score.
it consists of gene expression correlation scores by regression so it goes like features [0.9,0.8,..., 0.5](about 60) relevance score [0.4].
relevance score is something I want to predict
so there isn't much work about this. genes are about 20000 and I train with all pair so combination between gene pairs thats why I have so many instance of data.
Currently I'm using plain MLP and 4 layer was best for now but I have slight improvement from linear regression. and one more weird thing is that linear regression perform better than random forest regression or gradient boost tree regression models
since I have so many rows it takes so many time to train and evaluate