0
$\begingroup$

i am trying to translate this article into tflearn: http://iamtrask.github.io/2015/07/12/basic-python-network/

I have successfully created a linear neural network in tflearn:

import numpy as np
import tflearn

X = [[0, 0, 1],
    [0, 1, 1],
    [1, 0, 1],
    [1, 1, 1]]

Y = [[0, 1],
    [0, 1],
    [1, 0],
    [1, 0]]

Xtest = np.array([[1, 1, 0],
                  [0, 1, 0],
                  [1, 0, 1],
                  [0, 1, 1]])

# Build neural network
net = tflearn.input_data(shape=[None, 3])
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net)

# Define model
model = tflearn.DNN(net)
# Start training (apply gradient descent algorithm)
model.fit(X, Y, n_epoch=10000, batch_size=16, show_metric=True)

pred = model.predict(Xtest)
for i in range(4):
    print(pred[i][0])

the prediction for Xtest gives the correct answer: [1, 0, 1, 0]

Training Step: 10000  | total loss: 0.22091 | time: 0.002s
| Adam | epoch: 10000 | loss: 0.22091 - acc: 0.9491 -- iter: 4/4
--
0.9961615800857544
0.049626611173152924
0.9893295168876648
0.009963239543139935

However, when i make the matrix NON-linear:

X = [[0, 0, 1],
    [0, 1, 1],
    [1, 0, 1],
    [1, 1, 1]]

Y = [[0, 1],
    [1, 0],
    [1, 0],
    [0, 1]]

my prediction accuracy becomes 50/50 (because the matrix is no longer a linear relationship) and the predicted answer is wrong:

Training Step: 10000  | total loss: 0.69315 | time: 0.002s
| Adam | epoch: 10000 | loss: 0.69315 - acc: 0.5032 -- iter: 4/4
--
0.501151978969574
0.5012017488479614
0.49844005703926086
0.5015537738800049

how do i set this up correctly for the non-linear relationship?

$\endgroup$

1 Answer 1

0
$\begingroup$

You need to apply an activation function after your hidden layers, otherwise your network is just the softmax of a linear transformation.

For example, replace

net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)

with

net = tflearn.fully_connected(net, 32, activation='relu')
$\endgroup$
6
  • $\begingroup$ thanks, i tried that but im getting a max accuracy of 75% now $\endgroup$ Commented Jun 4, 2017 at 7:19
  • $\begingroup$ @Mark Markrowave Charlton works fine for me $\endgroup$ Commented Jun 4, 2017 at 7:28
  • $\begingroup$ can you send me your full code? what accuracy are you getting? $\endgroup$ Commented Jun 4, 2017 at 7:31
  • $\begingroup$ I literally just made the change in my answer, but almost any architecture will work for such an easy problem. I am getting 99.99-100% accuracy. $\endgroup$ Commented Jun 4, 2017 at 7:35
  • $\begingroup$ oh my goodness i must of messed something up, its working just fine now thanks so much for your help! $\endgroup$ Commented Jun 4, 2017 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.