I have been following This book on the fundamentals of NNs. It is currently outlining the MSE Cost function, and the Notation is tripping me up some.
$$ C(w, b) = \dfrac{1}{2n} \sum_x \vert\vert y(x)-a\vert\vert^2 $$
** Note x is tied with sum operator, LaTex was weird
** Note if any other notation is still confusing the book does its best to explain. I am in This section, just after the start.
I have some questions.
The book read that "The notation ∥v∥ just denotes the usual length function for a vector v" what exactly is the "length function" of a vector.
The book also denotes that y(x) is a column vector representing the desired output of the network with the input x while a is the actual output. I know you can subtract vectors, but how can you square one to get a single value. Or am I missing something.
Additionally, why do we divide by 2n rather than just n to get the mean value. (n being the number of training inputs you provide the network with.
Finally, I have been coding a Network with what I know, and for the time being I have been avoiding biases. As for now I am just using the sign activation function to simplify the process. How would the function change with only weights as an input? I have provided my code, if it helps with this problem. ** its is not complete now
from random import randint
from math import exp
import numpy as np
def map(f,mat):
arr = np.asarray_chkfinite(mat)
for i in range(len(arr)):
for k in range(len(arr[0])):
arr[i][k] = f(arr[i][k])
mat = np.asmatrix(arr)
return mat
def sigmoid(z):
return 1/(1 + exp(-z))
def sign(z):
if z > 0:
return 1
else:
return 0
class Point:
def __init__(self,x,y):
self.x = x
self.y = y
if x > y:
label = 1
elif x == y:
x += 1
label = 1
else:
label = 0
training_data = []
testing_data = []
for i in range(100):
p = Point(randint(0,500),randint(0,500))
training_data.append(p)
for i in range(100):
p = Point(randint(0,500),randint(0,500))
testing_data.append(p)
class NeuralNetwork:
def __init__(self,num_inputs,num_hiddens,num_outputs):
self.num_inputs = num_inputs
self.num_hiddens = num_hiddens
self.num_outputs = num_outputs
def initiate_weights(self):
self.weights_ih = np.zeros((self.num_hiddens,self.num_inputs))
for i in range(self.num_hiddens):
for j in range(self.num_inputs):
self.weights_ih[i][j] = randint(-5,6)
self.weights_ih = np.asmatrix(self.weights_ih)
self.weights_ho = np.zeros((self.num_outputs,self.num_hiddens))
for i in range(self.num_outputs):
for j in range(self.num_hiddens):
self.weights_ho[i][j] = randint(-5,6)
self.weights_ho = np.asmatrix(self.weights_ho)
def guess(self,inputs):
inputs_matrix = np.asmatrix(inputs)
inputs_matrix = np.reshape(inputs_matrix,(self.num_inputs,1))
weighted_ih = np.matmul(self.weights_ih,inputs_matrix)
self.activations_h = map(sign,weighted_ih)
weighted_ho = np.matmul(self.weights_ho,self.activations_h)
outputs = map(sign,weighted_ho)
return outputs
def train(self,inputs,labels):
n = self.num_inputs
error_sum = 0
guess = self.guess(inputs)
guess = np.asarray_chkfinite(guess)
for i in range(n):
** I would also appreciate any feedback on the code itself, I'm open to any suggestions
Thanks in advance, I know it's a lot