Do you need to scale Vectorizers in sklearn? I have a set of custom features and a set of features created with Vectorizers, in this case TfidfVectorizer.
All of my custom features are simple np.arrays (e.g. [0, 5, 4, 22, 1]). I am using StandardScaler to scale all of my featues, as you can see in my Pipeline by calling StandardScaler after my "custom pipeline". The question is whether there is a way or a need to scale the Vectorizers I use in my "vectorized_pipeline". Applying StandardScaler on the vectorizers doesn't seem to work (I get the following Error: "ValueError: Cannot center sparse matrices").
And another question, is it sensible to scale all of my features after I have joined them in the FeatureUnion or do I scale each of them separately (in my example, by calling the scaler in "pos_cluster" and "stylistic_features" seprately instead of calling it after the both of them have been joined), what is a better practice of doing this?
from sklearn.pipeline import FeatureUnion, Pipeline
from sklearn import feature_selection
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import StandardScaler
from sklearn.svm import LinearSVC

X = ['I am a sentence', 'an example']
Y = [1, 2]
X_dev = ['another sentence']

inner_scaler = StandardScaler()
# classifier
LinearSVC1 = LinearSVC(tol=1e-4,  C = 0.10000000000000001)

# vectorizers
countVecWord = TfidfVectorizer(ngram_range=(1, 3), max_features=2000, analyzer=u'word', sublinear_tf=True, use_idf = True, min_df=2, max_df=0.85, lowercase = True)
countVecWord_tags = TfidfVectorizer(ngram_range=(1, 4), max_features= 1000, analyzer=u'word', min_df=2, max_df=0.85, sublinear_tf=True, use_idf = True, lowercase = False)


pipeline = Pipeline([
    ('union', FeatureUnion(
            transformer_list=[

            ('vectorized_pipeline', Pipeline([
                ('union_vectorizer', FeatureUnion([

                    ('stem_text', Pipeline([
                        ('selector', ItemSelector(key='stem_text')),
                        ('stem_tfidf', countVecWord)
                    ])),

                    ('pos_text', Pipeline([
                        ('selector', ItemSelector(key='pos_text')),
                        ('pos_tfidf', countVecWord_tags)
                    ])),

                ])),
                ])),


            ('custom_pipeline', Pipeline([
                ('custom_features', FeatureUnion([

                    ('pos_cluster', Pipeline([
                        ('selector', ItemSelector(key='pos_text')),
                        ('pos_cluster_inner', pos_cluster)
                    ])),

                    ('stylistic_features', Pipeline([
                        ('selector', ItemSelector(key='raw_text')),
                        ('stylistic_features_inner', stylistic_features)
                    ]))

                ])),
                    ('inner_scale', inner_scaler)
            ])),

            ],

            # weight components in FeatureUnion
            # n_jobs=6,

            transformer_weights={
                'vectorized_pipeline': 0.8,  # 0.8,
                'custom_pipeline': 1.0  # 1.0
            },
    )),

    ('clf', classifier),
    ])

pipeline.fit(X, Y)
y_pred = pipeline.predict(X_dev)

 A: *

*Use StandardScaler after vectorization or not?


It's highly depends on the final estimator you want to use. For example, decision trees usually are not very sensitive to non-scaled features.
SVMs, especially with rbf kernels, are sensitive so it's better to try scaling.
As you pointed out sparse matrices can't be scaled with
with_centering=True
argument (because they lose their sparsity) but you can perform scaling using with_centering=False. Sklearn StandardScaler
The other solution would be sparse to dense transformation (if you have enough RAM).
from scipy import sparse
from sklearn.base import TransformerMixin, BaseEstimator

class Sparse2DenseTransformer(BaseEstimator, TransformerMixin):
    
    def fit(self, X, Y=None):
        return self
    
    def transform(self, X):
        return X.todense()

cc = Sparse2DenseTransformer().fit_transform( sparse.random(2000, 2000 ) )
sparse.issparse(cc) # False

You can add it to your pipeline right before StandardScaler.


*Is it sensible to scale all of my features after I have joined them in the FeatureUnion or do I scale each of them separately?


I believe it doesn't really matter.
