Why does this neural network in keras fail so badly? I am training a neural network using backpropagation and stochastic gradient descent in keras. However the network produces a graph that does not approximate the target funcion at all and I don't know why.
I've added the code here and a plot of the target function and resulting NN approximation below.
import math
import random

import matplotlib.pyplot as plt
import numpy
from matplotlib import pyplot

numpy.random.seed(7)
random.seed = 775
print(type(random.seed))
from keras.layers import Dense
from keras.models import Sequential
from keras.optimizers import SGD

THEANO_FLAGS = ""

r = Sequential()
numpy.array


def setup_nn():
    r.add(Dense(1, activation='sigmoid', input_dim=1, init='uniform'))
    r.add(Dense(50, activation='sigmoid', input_dim=1, init='uniform'))
    r.add(Dense(output_dim=1, activation='linear', input_dim=50))

    sgd = SGD(lr=0.05, decay=1e-6, momentum=0.9, nesterov=False)
    r.compile(loss='mean_squared_error', optimizer='sgd', metrics=['accuracy'])


def target_function(X):
    a = math.sin(X*3)
    return a*10


def trainRandomX(samplesize):
    X = []
    Y = []
    for j in range(0, samplesize):
        xj = random.random()
        X.append(xj)
        Y.append(target_function(xj))
    # X=numpy.array(X)
    # Y=numpy.array(Y)
    r.fit(X, Y, batch_size=100, nb_epoch=1)

    return


def testRandomX():
    X = [random.random()]
    Y = target_function(X[0])
    X = numpy.array(X)
    Ypred = r.predict(X, batch_size=1)
    error = Ypred[0][0] - Y
    print("error: ", error)
    # print(Ypred)
    return [X, Ypred[0][0]]


setup_nn()
plt.interactive(False)

# for i in range(0, 1):
trainRandomX(10000)

error = 0
X = []
Y = []
for i in range(0, 20):
    # error += abs(testRandomX())
    XY = testRandomX()
    X.append(XY[0][0])
    Y.append(XY[1])
pyplot.plot(X, Y, 'o')


def plotfunction():
    X = []
    Y = []
    for i in range(0, 100):
        x = i / 100
        X.append(x)
        Y.append(target_function(x))
    pyplot.plot(X, Y, '.')


plotfunction()

print("average error: ", error / 20)
plt.show()

Here the plot: the big dots are the neural network's approximation. Why don't they correspond to the target function better? 

 A: Here are a few observations:


*

*Your first layer of a single sigmoid neuron is a big bottleneck. Unless you are very lucky and the neuron is initialised to map your input onto the near-linear central part of the sigmoid, you end up with unnecessary information loss and vanishing gradient in the first layer right from the start. You maybe added this because a basic ANN is usually described to have 3 layers: input, hidden, and output, but the input layer is not actually modelled. It is better to think in the number of sets of weights are needed: input to hidden, and hidden to output.

*You only run a single epoch. That is not enough to learn the full function. At each epoch the weights are only updated a little bit to follow the local gradient, this needs to be repeated many more times to converge at the final result.

*In this case your batch size is fairly big, which means less updates are done per epoch. Usually smaller batch sizes can be more efficient, but this can differ per problem.
After removing the first layer, increasing nb_epoch to 100 and decreasing batch_size to 10, I get a much better result:

