How about viewing neural networks from an experimental point of view? Just because we created them doesn't mean that we're obligued to understand them intuitively. Or that we're not allowed to play with them in order to have a better grasp of what they're doing.

Here's a couple of thoughts I have on them:

- Structure: they are hierarchies. They are like trees that share inputs. The roots are the inputs and the leafs are the output layer. The closer the layer is to the outputs, the more relevant it is to them, the greater level of abstraction It contains.
- Functionality: they "play" with data, the modus operandi is to experiment with relationships in neurons (weights) until things "click" (the error margin is acceptable).

This is consistent with how we think. It's even consistent with how the scientific method operates. So by cracking this neural network thing we're also solving the general question of what knowledge representants.