1
$\begingroup$

I'm wondering what is the best way to handle sparse+non-sparse data in e.g. a Ridge regression using scikit learn.

Ridge can handle both sparse and nonsparse data.

Imagine something simple as a description (text) field that gets Count/Tdidf Vectorized (sparse), and an income continuous variable.

Now imagine that we have several other text fields and several other continuous variables.

What is the best way to model some continuous y variable?

I've considered making two separate models (one using sparse data, one using non-sparse) and somehow trying to combine.

I've also considered using PCA to make the sparse data into a "handleable" amount of continuous features.

How do you usually solve this issue?

Note: the continuous variables would have many unique values (and you'd lose power anyway when converting continuous to bins), and the text fields might end up having like a million features, thus not able to be dense.

$\endgroup$
1
$\begingroup$

You can have a look at : Feature Engineering and Classifier Ensemble for KDD Cup 2010 where the authors generated different models (for sparse and dense features) and combined it to obtain good perf (they actually won the KDD cup).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.