1
$\begingroup$

I am a newbie in Machine learning and I am writing a small code for Perceptron. This is the first time I am writing code in Python. I have four training data points (X). As they are used for supervised learning so, each data point has its corresponding correct output pair (D). I have implemented SGD and used generalized Delta rule (wij ← wij + α δixj). I have trained my perceptron 10,000 times (epochs= 10,000).
Although everything looks fine to me, I don't get the right results when I test it with test values. I need some suggestions so that I can improve my results on test data. P.S. How can I improve this code?

Code

import numpy as np 

def sigmoid(x):
return 1 / (1 + np.exp(-x))    

def Delta_SGD(W, X, D):
 N = 4
 for x in range(N):

    v1 = np.dot(X[x][0], W[0])
    v2 = np.dot(X[x][1], W[1])
    v3 = np.dot(X[x][2], W[2])
    #weighted sum
    V = v1+v2+v3

    #output of neuron
    y = sigmoid(V)

    #error 
    e = D[x] - y

    #derivative of sigmoid(y)
    delta = (y)*(1-y)*e

    #Delta rule
    DW = alpha*delta*X[x]

    #updated weights
    W[0] = W[0] + DW[0]
    W[1] = W[1] + DW[1]
    W[2] = W[2] + DW[2]

return W

#input data points
X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1] ])

#Correct output pairs
D = np.array([[0,0,1,1]]).T

#learning rate
alpha = 0.9

#random weights
W =  2*np.random.random((3,1)) - 1

#10000 epochs
for epoch in range(10000):
 W = Delta_SGD(W, X, D)
 print(epoch)

#Final weights after all epochs
print("Final weights are \n", W)

#testing network
N = 4
for x in range(N):

 v1 = np.dot(X[x][0], W[0])
 v2 = np.dot(X[x][1], W[1])
 v3 = np.dot(X[x][2], W[2])

 V = v1+v2+v3
 y = sigmoid(V)
 print("output of neuron is \n ", y)
$\endgroup$
  • $\begingroup$ This looks more like a stack overflow question - this section of SE is focused on the theory and decision making aspects of statistics - the what and why. Code questions are supported in SO with an army of statistical programmers. If still think CV is more relevant please read stats.stackexchange.com/help/on-topic and update your question to make it more appropriate $\endgroup$ – ReneBt Oct 19 '18 at 8:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.