3
$\begingroup$

I want to apply neural network as an auto associative memory. So, the desired output is equal to the input. I would apply Hebbs rule to train the network.

I have a pattern in the form

Sample1 =  [1 1 1 1 1 1 -1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 1 ]';

The length d = 30. I have a set of p samples stored in a database, Database, X = {Sample1,Sample2,....,Sample_p}

But I have some conceptual problem in understanding what determines the input to the neural network -- will it be all samples (example) or each sample /example? Would there be $p$ input neurons or $d$ neurons? In general, what is meant by number of inputs and number of outputs?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

In general, an example from the sample set is an input. :) This means that since you represent samples with $d$ units of data, your network will have $d+1$ input units: $d$ for the sample you want to process with the net, and one additional for the bias unit (that is always equal to $1$).

When you train the network, you do it with respect to all the samples in the training set. You may use Gradient Descent, or a metaheuristic optimization algorithm such as Genetic Algorithm, Particle Swarm Optimization, etc.

I've never worked with auto associative neural networks, so I don't know if there are specific ways to train this particular type of networks, but you can always start with traditional approaches and go for more complex if they turn insufficiently good.

I hope I helped a bit. :)

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you very much. Would it be possible to also say if instead of neural network representation for learning the weights, if I am using a channel whose impulse response is modeled as moving average, then would the input to the channel be d? But as you say that the training is performed for all examples, then the input to the channel should be p examples?I want to relate the idea to signal processing where the input source is the feature vector that passes through a channel. $\endgroup$
    – Sm1
    Commented Jul 11, 2016 at 21:45
  • $\begingroup$ Parameter estimation of the channel's impulse response is performed and input is also estimated. The input to the channel is the feature vector just as in case of neural network. How does it relate to single input single output channel or multiple input channel? $\endgroup$
    – Sm1
    Commented Jul 11, 2016 at 21:48
  • $\begingroup$ Sorry, I'm not familiar with signal processing. But if it helps, the networks have multiple matrices of unknown parameters that we need to train in order to achieve good performance. We do it by optimizing the function $E = \frac{1}{2}\sum_{i=1}^{p}||out_i-y_i||^2$, where $out$ is the $i$th output vector and $y_i$ is the actual vector that corresponds to the $i$th example in the set. Training essentially includes running the net for each example you want to process (that is, for $p$ times). $\endgroup$
    – Milos
    Commented Jul 12, 2016 at 0:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.