0
$\begingroup$

I am trying to build a pattern recognition neural network model.

I was told that the target for pattern1 is 1, pattern2 is 0, and pattern3 is -1. However, I am not sure how to create my weight, bias and target vectors?

My input vectors are the following:

pattern_1 = [1 1 1 -1];
pattern_2 = [1 -1 -1 1];
pattern_3 = [1 -1 1 -1];

How do I write the target vector for the patterns? Is it:

weight = [0 0 0];
bias = [1 1 1];
target = [1 0 -1];

I will eventually want to perform:

a = weights * pattern_1
error = target - a

But I am unsure what sized matrix my target vector needs to be, given the fact that I was just told that it is either 1, 0 or -1.

$\endgroup$
1
$\begingroup$

You can think of the bias as an additional feature whose value is 1. Therefore with the bias, your input vectors are,

pattern_1 = [1 1 1 -1 1];
pattern_2 = [1 -1 -1 1 1];
pattern_3 = [1 -1 1 -1 1];

or you can write the input vectors as one matrix,

$$ X = \begin{bmatrix} 1& 1 & 1&-1&1\\ 1& -1 & -1&1&1\\ 1& -1 & 1&-1&1 \end{bmatrix} $$ where each row corresponds to each pattern vector.

Note that I added the bias values at the end of each input vector.

Your initial weight vector is [0 0 0 0 0] or a vector of random values. The weight vector should have a weight for every feature of your input vector.

Since the target values are in the set $\{-1,0,1\}$, it is a multi-classification problem. You might want to use the softmax objective function.

You can write the target values (denote it as $y$) as a matrix using one-hot encoding. The target matrix is

$$ y = \begin{bmatrix} 0& 0 & 1\\ 0& 1 & 0\\ 1& 0 & 0 \end{bmatrix} $$ where each row corresponds to a target value of a pattern vector.

Index $j$ of a row $i$ of $y$ represents the target value of pattern (or sample) $i$. There are three indices for each row vector. Index 0 represents target -1, Index 1 target 0, and Index 2 target 1.

Now your dataset consists of the input matrix $X$ and the target matrix $y$. To optimise your weight vector $w$ you need to choose an objective function. You can start with the softmax objective function explained here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.