# Training a neural network (R) [duplicate]

I'm working on a neural network with one hidden layer. So far I've implemented the algorithm, and I've been able to numerically verify the partial derivatives I get from back propagation. My problem is when I try to train the network by updating the weights. No matter what I do it doesn't seem like I can get my estimates to come close to the actual values.

The data I'm using is from the function:

$$f(x) = \frac{\sin(x)}{x}$$

On a side note, I haven't added bias parameters because I couldn't get the dimensions to fit in back propagation once I added the bias in forward prop. Not sure if this is an issue since I understood the bias parameters as a safeguard against inputs equal to 0?

theta <- function(a){
a / (1+abs(a))      # Here we apply the alternative sigmoid function as our
# non-linearity.
}
theta.prime <- function(a){
1 / (1+abs(a))^2
}

x <- c( 5.949110, -1.036600,  3.256780,  7.824520, -3.606010,  3.115640, -7.786960,
-7.598090,  2.083880,  3.983000,  8.060120,  7.879760, -2.456670,
-2.152720,  3.471950,  3.567960, -4.232630,  6.831610, -9.486860,  8.692330,
-1.551860,  0.917305,  4.669480, -7.760430,  2.835410)
y <- c(-0.10804400,  0.78264000, -0.05313330,  0.13484700, -0.05522470, -0.05758530,
0.19566100,  0.13846000,  0.43534100, -0.16861400,  0.10625000,
0.08427310,  0.27012900,  0.44004800, -0.00880575, -0.10711400, -0.18671100,
0.01158470,  0.02767190,  0.06319830,  0.61802000,  0.87124300,
-0.25668100,  0.06160800, 0.10575700)

neuralnet <- function(x,y,theta,theta.prime,neurons){

w1 <- t(matrix(rnorm(neurons,0,.01),neurons,1))
w2 <- matrix(rnorm(neurons,0,.01),neurons,1)
E <- 1
i <- 0
for(i in 1:1000){
### Forwardpropagation ###
s2 <- x%*%w1
a2 <- apply(s2,c(1,2),theta)
s3 <- a2%*%w2
yhat <- apply(s3,c(1,2),theta)
### Error function ###
E <- sum((yhat-y)^2)/length(x)

### Back Propagation ###
delta3 <- (2*(yhat-y)) * apply(s3,c(1,2),theta.prime)
djdw2 <- t(a2) %*% delta3
delta2 <- delta3 %*% t(w2) * apply(s2,c(1,2),theta.prime)
djdw1 <- t(x)%*%delta2

### Update the weights ###
w1 <- w1 - 0.01*djdw1
w2 <- w2 - -0.01*djdw2
i <- i+1
}
print(i)
print(E)
return(yhat)
}

yhat <- neuralnet(x,y,theta,theta.prime,2)

• Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. Jan 1 '17 at 18:38