7
$\begingroup$

People talk about how neural network should learn "disentangled representation" rather than "distributed representation" so that a deep learning model is more interpretable and understandable. What are the definition of these two terms and any particular example?

$\endgroup$

1 Answer 1

9
$\begingroup$

Let's say the following vectors are respectively representations for a ball: [1,0,0,0] and a car: [0,1,0,0]

In this representation a single neuron learns the meaning of a ball or a car without having to rely on other neurons. This is a disentangled representation, which is meant to facilitate the understanding of artificial neural networks.

This in contrast to distributed representations, for example, a ball: [0.1,-0.02,0.45,0.06] and a car: [-0.78,-0.1,0.83,0.01]. In this case, an object is represented by a particular location in the vector space. This type of representation is for example the outcome of the word2vec algorithms.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.