0
$\begingroup$

I have two different feature vectors of completely different scale, which are to be used as training data for machine learning algorithm.

When I concatenate them,

should I scale and normalize them separately first and then concatenate?

Or should I concatenate them first and scale-normalize them together?

$\endgroup$
2
$\begingroup$

This is too broad question and must be answered carefully. What are your features? If you have categorical features, you are better to encode it in one-hot manner and then you may circumvent scaling problems. Also, it depends on a classifier you want to use downstream : SVM will not tolerate badly scaled and uncentered data, but XGBoost will do ( to some degree )

Without much details it is hard to know whether rescaling your data will improve the performance of your algorithms before you apply them. If often can, but not always.

A good tip is to create rescaled copies of your dataset and race them against each other using your test harness and a handful of algorithms you want to spot check. This can quickly highlight the benefits (or lack there of) of rescaling your data with given models, and which rescaling method may be worthy of further investigation.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ One side is a histogram-like features, and the other is features from convolutional neural network.. Right now, I'm concatenating them first, and then scale-l2 normalizing, but since those features are of different scales, I was wondering whether they should be scale-normalized first prior to getting concatenated.. $\endgroup$ – cosmiccapsule May 18 '16 at 14:04
  • $\begingroup$ It depends on what you do next. Maybe you better build two separate models and then ensemble them using blending, for example ( mlwave.com/kaggle-ensembling-guide ) and that could be better than concatenating features. Honestly, I would just give it a try and figure out. $\endgroup$ – Vast Academician May 18 '16 at 14:12
0
$\begingroup$

As mentioned above you could train two separate classifiers on both the data. This is useful because say you have more faith in the CNN features than histogram based features then you give that classifier's prediction more weight in the final output. Once you concatenate the features you are unable to use make use of this prior knowledge. If both features sets are equally informative with similar noise then concatenation should work fine as well.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.