I am using the hashing trick for a machine learning problem:


In this particular problem there are a large number of categorical variables (some with high cardinality), and as a result, hash encoding categorical variables in the feature space has helped to improve the model.

However, as an input to hash encoding, you are required to specify the dimensionality of the vector space you wish to return from feature hashing, for example (in this python code snippet):

h = FeatureHasher(n_features=50,input_type="string")

Where n_features is the dimensionality of the returned matrix. My question is as follows:

How can you determine the optimum dimension to return from hashing? I am unable to find any literature on this particular topic.

I could run a sensitivity over the dimension but this would be rather time consumping. Any advice would be greatly appreciated.

  • $\begingroup$ there is no 'optimum': basically you aim for the maximum that will allow your data to eg fit in memory. collisions are in theory bad - if they have a positive effect its just an artifact $\endgroup$ – seanv507 Oct 3 '18 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.