2
$\begingroup$

In Neural Network Architectures I understand that lower dimensional mappings(for example mapping and input to a space of lower dimension) can serve the purpose of decreasing dimensionality while retaining the important information from the input. My questions is in what circumstances would we want to use a higher dimensional mapping. For example what is the overall impact of having a hidden layer with more neurons than the layer that came before it.(Is it because we are able to learn more granular features that are not as salient in the original input)

I am trying to understand this so that I generally understand where in my neural networks I upscale the input data and where I start downscaling the data.

$\endgroup$

1 Answer 1

4
$\begingroup$

In general, not just in neural networks, one common use of higher dimensional mappings is to make the data separable in the high dimensional space. For example, the whole idea of kernel SVM's is based on this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.