I have a column X which contains occupation/profession as an independent variable as free text, which is very much correlated with a continuous dependent variable.
What techniques do you usually use to 'bin' such free text variables for regression?
I am thinking about breaking the free text strings down into quadgramms (4 characters) and thus create a high dimensional matrix. I could then apply word embedding/SVD etc. to create a low-ish dimensional matrix. The matrix could then be clustered and the cluster membership would represent the binned value.