# How to improve the accuracy of a neural network model?

I am attempting to create a neural network model using R that estimates the sine function for a given x variable, i.e. sin(x). Dataset has 5000 observations with x iterating by 0.05 each time:

However, when I estimate using the model, my error is quite high at 219.2 and my prediction remains fairly constant at a value of 0.48 or thereabouts, while the actual value deviates sharply between 0 and 1:

The steps I am taking to estimate the model are as follows:

1) I am scaling my sin and x variables using max-min normalization:

sinscaled=(sin-min(sin))/(max(sin)-min(sin))
xscaled=(x-min(x))/(max(x)-min(x))


2) I am then fitting my training and test data:

trainset <- maxmindf[1:3500, ]
testset <- maxmindf[3501:5000, ]


3) The neural network is fitted as below, and the resulting output tested:

#Neural Network
library(neuralnet)
sinscalednet <- neuralnet(sinscaled ~ xscaled, data=trainset,
hidden=c(2,1), linear.output=FALSE, threshold=0.01)
sinscalednet$result.matrix plot(sinscalednet)  Note that while I am using 2 hidden layers in this instance, the error is remaining consistent even when the number of hidden layers are being manipulated: 4) Resulting output is tested and accuracy obtained: #Test the resulting output temp_test <- subset(testset, select = c("xscaled")) head(temp_test) sinscalednet.results <- compute(sinscalednet, temp_test) #Accuracy results <- data.frame(actual = testset$sin, prediction =
sinscalednet.results$net.result) results roundedresults<-sapply(results,round,digits=0) roundedresultsdf=data.frame(roundedresults) attach(roundedresultsdf) table(actual,prediction)  Would be grateful for any advice on steps I need to take to improve this model. I am experimenting by trying to modify the hidden layers, but the error is remaining consistently high. • did you scale your training and test inputs together? Because if so, then$x_i > x_j\forall i \in T_e$,$j \in T_r$where$T_e$is the test index set and$T_r\$ is the training index set, so you're predicting on data well beyond the range of anything seen in training. You should randomly partition your data into training and test sets instead. – jld May 17 '17 at 21:32
• Quite possibly - an oversight on my part. Will check and run again. – Michael Grogan May 17 '17 at 23:55

What @Chaconne mentioned in the comments is quite important. You should first shuffle your training set and then split the array into chunks.

But I rewrote your code to the Javascript neural network library Neataptic, and i'm not having any issues. This is the code I wrote:

// Create the network
var network = new Architect.Perceptron(1,2,1);

// Create training set
var set = [];
for(var i = 1; i <= 5000; i++){
var x = i * 0.05;
x = x / (5000 * 0.05);

var y = Math.sin(x);
y = (y + 1) / 2;

set.push({ input: [x], output: [y]});
}

var trainingSet = set.splice(0, 3500);
var testSet = set;

// Train the network
var training = network.train(trainingSet, {
iterations: 500, // it will never reach this
error: 0.0001,
rate: 0.01,
cost: Methods.Cost.MSE
});

var testError = network.test(testSet, Methods.Cost.MSE);


Run the code yourself here (open console)!

I'm getting these kind of results somewhat consistently:

training error: 0.00008
test error:  0.00133


So dropout would fix the large gap, but my point is, something else must be wrong in your model. Consistently getting 0.48 as prediction seems like the network has never learned anything.

What is your training error? What is your test error? How long are you training for (iterations).

• I have rearranged the data, but still get the same error. Moreover, when I run a neuralnet for an equal number of observations (for both training and test data), I get an error of 157 for 24 steps, and an error of 155 for 32 steps. Yes, something is wrong in the model but I can't quite work out what. In other neural network models I've run, I haven't encountered this problem. Even setting linear.output = TRUE has not alleviated the issue. – Michael Grogan May 18 '17 at 13:16