1
$\begingroup$

when we are applying feature hashing in sklearn it asks us what should be the dimensions of feature required for us. If we decrease too much there will be more collisions which are not good. And we cannot make collisions to stop if we want to decrease cardinality. So, how to select a good value for the problem. Is there any trick to find the optimum value according to the situation??

$\endgroup$
1
$\begingroup$

Feature hashing uses hash functions that are designed to be fast and fill the space of hash values uniformly given the inputs, but they don't do anything to group the values together in any meaningful way. Moreover, that there are many different hash functions, starting from generic like the Python build-in's, SHA-1, or MD5, ending on more specialised ones like this, that would give you different results. What follows, there is no simple way of deciding a priori on the size of the hashing table. Hashing collisions are certain to happen, so it is a question is on how they would affect your results. Usually this is something that you would verify empirically, by trying different sizes and looking at test set performance and quality of the results, so basically this is another hyperparameter to tune. Notice also that it is not only about making it as 'big as possible', since using hashing trick is known to have regularizing effect on the model, so in fact, in some cases, can decrease the overfitting and improve your results. You can check the papers by Weinberg et al (2009) and Shi et al (2009) who discuss this in greater detail.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.