2
$\begingroup$

I'm trying to find patterns in a large dataset using the neuralnet package.

My data file looks something like this (30,204,447 rows) :

id.company,EPS.or.Sales,FQ.or.FY,fiscal,date,value

000001,EPS,FY,2001,20020201,-5.520000
000001,SAL,FQ,2000,20020401,70.300003
000001,SAL,FY,2001,20020325,49.200001
000002,EPS,FQ,2008,20071009,-4.000000
000002,SAL,FY,2008,20071009,1.400000

I have split this initial file into four new files for annual/quarterly sales/EPS and it is on those files that I want to use neural networks to see if I can use the variables id.company, fiscal and date in the case below to predict the annual sales results.

To do so, I have written the following code:

dataset <- read.table("fy_sal_data.txt",header=T, sep="\t") #my file doesn't actually use comas as separators

#extract training set and testing set
trainset <- dataset[1:1000, ]
testset <- dataset[1001:2000, ]

#building the NN
ann <- neuralnet(value ~ id.company + fiscal + date, trainset, hidden = 3,
             lifesign="minimal", threshold=0.01)

#testing the output
temp_test <- subset(testset, select=c("id.company", "fiscal", "date"))
ann.results <- compute(ann, temp_test)

#display the results
cleanoutput <- cbind(testset$value, as.data.frame(ann.results$net.result))
colnames(cleanoutput) <- c("Expected Output", "NN Output")

head(cleanoutput, 30)

Now my problem is that the compute function returns a constant answer no matter the inputs of the testing set.

     Expected Output   NN Output
1001     2006.500000 1417.796651
1002     2009.000000 1417.796651
1003     2006.500000 1417.796651
1004     2002.500000 1417.796651

I am very new to R and its neural networks packages but I have found online that some of the reasons for such results can be either:

  • an insufficient number of training examples (here I'm using a thousand ones but I've also tried using a million rows and the results were the same, only it took 4h to train)

  • or an error in the formula.

I am sure I'm doing something wrong but I can't seem to figure out what.

$\endgroup$
9
  • $\begingroup$ I don't know that package. One thing is that you have set the number of hidden elements to be 3. The rule of thumb is hidden layer should be 4x size of input layer for single layer nets. (I wish i remember who to cite for this). Your hidden layers is smaller than your input layer. That would not however explain constant output. $\endgroup$ Jun 19, 2015 at 13:51
  • $\begingroup$ I had read that one layer was enough for most problems. And the same person said that 'the optimal size of the hidden layer is usually between the size of the input and size of the output layers'. Jeff Heaton, author of Introduction to Neural Networks in Java. I kind of blindly followed this recommendation so I can always try to add some neurons to that hidden layer. Edit: raising the size of the hidden layer from 3 to 10 caused the network to fail to converge (even though I didn't specify a max number of iterations) $\endgroup$
    – OKS N.
    Jun 19, 2015 at 14:14
  • $\begingroup$ Edit: Setting it back to 3 doesn't make it "right", now it just won't converge no matter what. $\endgroup$
    – OKS N.
    Jun 19, 2015 at 14:22
  • 1
    $\begingroup$ It is proven that one layer (with activations of the form $A\sigma(Wx+b)$ for any bounded continuous, differentiable, monotonic $\sigma$) can approximate to arbitrary accuracy (ie as good as one wants) any continuous function, given sufficient width. What that width is we don't know. Nor that any training method can achieve that performance (Well monte carlo will do it in infinite time), let alone what method to use. Modern thinking -- since 2006 (the year after that book was published) -- (See paper by Bengio), is that Deeper Nets are better, but they have there own giant pile of issues. $\endgroup$ Jun 19, 2015 at 15:07
  • $\begingroup$ I will keep my eye out for the reference that said 4x input width. I'm dubious of it now. I might have misremembered $\endgroup$ Jun 19, 2015 at 15:10

2 Answers 2

0
$\begingroup$

I think the problem might be the learning rate rather than the size of the hidden layer. Too slow and training has no effect on some weights; too fast and the weights over-shoot and then take a long time to get back to sensible values. Maybe experiment with the "algorithm" argument (which controls the learning rate).

$\endgroup$
6
  • $\begingroup$ Thank you, I will definetly try that. Is there a rule of thumb for setting the learning rate ? Because I don't even know what the default value is (I believe the default algorithm with neuralnet is the resilient backprop with weight backtracking). I tried to re run the code above to look at the weights of the network but now it says that the algorithm did not converge in x of y repetition(s) with the stepmax which it didn't before... it seems so random ! $\endgroup$
    – OKS N.
    Jun 22, 2015 at 11:11
  • $\begingroup$ I changed the training parameters to: ann <- neuralnet(value ~ id.company + fiscal + date, trainset, hidden = 3, lifesign="minimal", algorithm="backprop", learningrate=0.01, threshold=0.01) which returns "error in if (reached.threshold < min.reached.threshold) : missing value where TRUE/FALSE needed. There seem to be a problem with the comparison but I don't even know how to specify those two arguments. $\endgroup$
    – OKS N.
    Jun 22, 2015 at 11:27
  • $\begingroup$ Yep.,the error messages in R can be cryptic.If it wasn't free, you'd demand your money back. I'd guess reducing the threshold argument might work. Or at least might generate a different error. $\endgroup$
    – nsweeney
    Jul 2, 2015 at 11:12
  • $\begingroup$ I tried different values for my threshold but none allowed it to work. It still won't converge or still gives a constant answer when it randomly does. As for the standard backpropagation and the error described in my previous post, I'm still clueless and I've had to go back to neuralnet's default algorithm. $\endgroup$
    – OKS N.
    Jul 3, 2015 at 13:17
  • $\begingroup$ I've had trouble getting a neural net started as well and that's when I wrote the code myself. When I had a look at the output you put up, it looks like the learning rate is still too high. Values of over 700 from a hidden node suggest that the learning rate is too high which causes instability in the weights. $\endgroup$
    – nsweeney
    Jul 4, 2015 at 11:26
-1
$\begingroup$

You can add this to your neuralnetwork

  • threshold = 0.5,
  • learningrate = 0.01,
  • act.fct = "tanh",
  • stepmax = 100000,
  • linear.output = T

You can vary the values accordingly. I had same problem and it worked for me

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.