Can't reproduce results using foreach and RSNNS I'm trying to train multiple Multi-Layer-Perceptrons using foreach and mlp(...) from the RSNNS package. As I ran into problems reproducing the results generated by mlp() and foreach in a parallel environment, I created a minimal example without a parallel backend:
library(foreach)
set.seed(42)
a <- foreach(i=1:10) %do% { runif(3) }
set.seed(42)
b <- foreach(i=1:10) %do% { runif(3) }
identical(a, b)
TRUE

This works as intended. a and b are identical. The next example is a modified version of the RSNNS demo.
set.seed(42)
library(foreach)
library(RSNNS)
data(iris)

#shuffle the vector
iris <- iris[sample(1:nrow(iris),length(1:nrow(iris))),1:ncol(iris)]

irisValues <- iris[,1:4]
irisTargets <- decodeClassLabels(iris[,5])

iris <- splitForTrainingAndTest(irisValues, irisTargets, ratio=0.15)
iris <- normTrainingAndTestSet(iris)

hiddenUnits = c(5,10)

set.seed(42)
trainedModels <- foreach(i=hiddenUnits, .packages='RSNNS') %do% {

        mlp(iris$inputsTrain, iris$targetsTrain, 
            size = i,
            maxit = 50,
            initFunc = "Randomize_Weights", initFuncParams = c(-0.3, 0.3),
            learnFunc = "BackpropWeightDecay",
            learnFuncParams = c(0.4, 0, 0, 0),
            updateFunc = "Topological_Order", updateFuncParams = c(0),
            hiddenActFunc = "Act_Logistic",
            shufflePatterns = F, linOut = FALSE,
            inputsTest = iris$inputsTest, targetsTest=iris$targetsTest,
            pruneFunc = NULL, pruneFuncParams = NULL)}

set.seed(42)
trainedModels2 <- foreach(i=hiddenUnits, .packages='RSNNS') %do% {

        mlp(iris$inputsTrain, iris$targetsTrain, 
            size = i,
            maxit = 50,
            initFunc = "Randomize_Weights", initFuncParams = c(-0.3, 0.3),
            learnFunc = "BackpropWeightDecay",
            learnFuncParams = c(0.4, 0, 0, 0),
            updateFunc = "Topological_Order", updateFuncParams = c(0),
            hiddenActFunc = "Act_Logistic",
            shufflePatterns = F, linOut = FALSE,
            inputsTest = iris$inputsTest, targetsTest=iris$targetsTest,
            pruneFunc = NULL, pruneFuncParams = NULL)}

identical(trainedModels, trainedModels2)
FALSE

The results aren't identical. Is there a way to make the results reproducible? The goal would be to run the code in parallel using the package doParallel. But I can't even get the same results in sequential mode.
Edit: As shufflePatterns = F, the only random component in the models should be the random initialisation of the weights. It seems like set seed(x) has no effect on the generation of the random starting weights.
Edit2: Did some research and found this on the github site of RSNNS: 
Changelog for version 0.4-6 (22-12-2014):
...


*

*now using R's random number generator instead of the system's one


Updated to the new version but I still can't reproduce results, even without foreach loops:
set.seed(42)
trainedModels <- mlp(iris$inputsTrain, iris$targetsTrain, 
                      size = 5,
                      maxit = 50,
                      initFunc = "Randomize_Weights",initFuncParams = c(-0.3, 0.3),
                      learnFunc = "BackpropWeightDecay",
                      learnFuncParams = c(0.4, 0, 0, 0),
                      updateFunc = "Topological_Order", updateFuncParams = c(0),
                      hiddenActFunc = "Act_Logistic",
                      shufflePatterns = F, linOut = FALSE,
                      inputsTest = iris$inputsTest, targetsTest=iris$targetsTest,
                      pruneFunc = NULL, pruneFuncParams = NULL)
set.seed(42)
trainedModels2 <- mlp(iris$inputsTrain, iris$targetsTrain, 
                      size = 5,
                      maxit = 50,
                      initFunc = "Randomize_Weights",initFuncParams = c(-0.3, 0.3),
                      learnFunc = "BackpropWeightDecay",
                      learnFuncParams = c(0.4, 0, 0, 0),
                      updateFunc = "Topological_Order", updateFuncParams = c(0),
                      hiddenActFunc = "Act_Logistic",
                      shufflePatterns = F, linOut = FALSE,
                      inputsTest = iris$inputsTest, targetsTest=iris$targetsTest,
                      pruneFunc = NULL, pruneFuncParams = NULL)
identical(trainedModels, trainedModels2)
FALSE

 A: Problem got solved. Detailed explanation by Christoph Bergmeir, maintainer of the package:
Hi,
the latest version 0.4-6 that was published yesterday had some changes
about the random number generator that should solve exactly the
problem you have. They have not been extensively tested but should work.
Before 0.4-6, you had to use the function setSnnsRSeedValue()
additionally to set.seed(). This shouldn't be necessary anymore, and
just calling set.seed() as you do it should be enough.
The problem with your example on stackoverflow is the following: You
check if the models are identical but they obviously are not. That is
because the models are C++ objects and every time you train a model
you will get a new such object. Look at:
trainedModels[[1]]$snnsObject
trainedModels2[[1]]$snnsObject

You'll see that they point to different C++ objects.
However, the content of both models should be exactly the same. You
can check this, e.g., with
summary(trainedModels[[1]])

Doing something like:
res1 <- summary(trainedModels[[1]])
res2 <- summary(trainedModels2[[1]])

cat(res1)
cat(res2)

identical(res1, res2)

res <- rep(FALSE, length(trainedModels))
for(i in 1:length(trainedModels)) {
  res[i] <- identical(summary(trainedModels[[i]]),
  summary(trainedModels2[[i]]))
}
res

I get here as a result that the models from your example are actually
the same. But take into account that the summary also shows the time,
so if you wait too long between executions you will actually get FALSE
as result.
I hope this resolves your problem and you can reproduce what I've
done. Another good idea is to just use the networks for prediction and
see if they produce the same outcomes, which is what I have done in my
tests for yesterday's release.
