As input into a simple neural network multi-class classifier, I am considering using a variation of the standard one-hot sparse matrix to represent categorical variables. Instead of each element represented by all zeroes and a single one value, I propose using TF-IDF, which would result in a sparse matrix with all zeroes and a single positive float value (0,inf) in each row.

What would be the effect of this change on the neural network classification results? Would elements with larger positive values in their sparse representations be given greater importance in the network than those with smaller values? My intuition is that they would, as these values are directly multiplied by the weights, but I'm not certain.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.