Does the number of outputs of a neural network have to be smaller than the number of inputs? This seems to be the case in most of the applications that I have seen . At the same time, I am not sure how this might be the case in language translation models. It is not guaranteed that the output (the translation of a text) would be the same or smaller length as the input text. It can be longer or shorter.
1
-
$\begingroup$ You can imagine a toy example of this where you are classifying 32x32 pixel images (1024 pixel inputs) into, say, 2000 different categories. That's a somewhat awkward example, but I'm sure there are expansions of that idea that actually provide value in certain domains. $\endgroup$– userAug 26, 2020 at 4:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
No! There is no limit whatsoever on the size of the output relative to the size of the input.
But in most cases, a higher number of outputs is not necessary at all.
In the case of language translation models: this is either done fragment by fragment, or the output has a fixed maximum size (e.g. 2x input size).
