0
$\begingroup$

This is my first post in CrossValidated hence please let me know if I may have inadvertently violated forum rules.

I am working with nnet using Caret in R and when I am running experiments using the tuning grid I am somehow not able to get any results with size = 8 and above.

enter image description here

My code is as follows:

   set.seed(seedVal)

  ### creating a grid of tuning parameters
  nnetTunegrid <- expand.grid(.size = seq(min_tune,max_tune,step_tune),
                              .decay = seq(0,4,0.125))

  # set seeds array for cross validation
  seeds <- setSeeds(cv_count, cv_repeats, nrow(nnetTunegrid), seedVal)

  # Define cross-validation experiment
  numFolds = trainControl(method = "cv",
                          number = cv_count,
                          #repeats = cv_repeats,
                          seeds = seeds,
                          classProbs = TRUE,
                          summaryFunction = twoClassSummary)

  registerDoParallel(cores = 6)

  nnetFit <- train(x = train_matrix, y = catg_labels,
                   method = "nnet",
                   preProc = preProcessing,
                   trControl = numFolds,
                   tuneGrid = nnetTunegrid,
                   maxit = 500, # max iterations for nnet only
                   metric = metricVal)

My data set has 150 features and I am using nnet to do binary classification.

Any help or pointers to resolve this problem would be appreciated!

Thanks Ian

$\endgroup$
2
$\begingroup$

The R nnet package cannot work when the number of estimated weights is greater than the number of observations. The number of weights is: H*(P+1) + (H+1) where H is the number of hidden units (size=8) in the layer and P is the number of predictors (150). Maybe this is the problem. I think this restriction is general for other ANN packages and software

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi Jesus, thanks for this insight! I will test this by changing the no. of predictors. $\endgroup$ – Ian Lo Jan 21 '16 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.