3
$\begingroup$

I'm trying to build a regression model with ANN with scikit-learn using sklearn.neural_network.MLPRegressor. I have a 1000 data samples, which I want to split like 6:2:2 for training:testing:verification. For network learning, I want to perform 100 steps with 100 mini batches each.

My questions are:

  1. How should I set parameter batch_size. How can I get some insight on iterating over mini batches? How can I know which indexes were used in each iteration? Does it update model weight with each mini batch?
  2. What data and what cost function is applied be default in this method?
  3. Does parameter max_iter set number of iterations (epochs?) to 100? How my data are used? Can I use here some K-Fold technique to improve results?

My code:

from sklearn.model_selection import train_test_split    
from sklearn.neural_network import MLPRegressor

X_train, X_test, y_train, y_test =  train_test_split(
        X_scaled, y_data, test_size=0.20, random_state=1)

rgr = MLPRegressor(hidden_layer_sizes=(100, ), 
                   activation='logistic', 
                   solver='sgd', 
                   alpha=0.0001, 
                   batch_size=8, 
                   learning_rate='constant', 
                   learning_rate_init=0.001, 
                   power_t=0.5, 
                   max_iter=100,
                   shuffle=True, 
                   random_state=None, 
                   tol=0.001, 
                   verbose=True, 
                   warm_start=False, 
                   momentum=0.9, 
                   nesterovs_momentum=True, 
                   early_stopping=False,
                   validation_fraction=0.25, 
                   )

for i in list(range(100)):
    rgr.partial_fit(X_train, y_train)

y_predicted = rgr.predict(X_test)`

Is there anything else what could be worth considering in this code?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.