Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have two different feature vectors of completely different scale, which are to be used as training data for machine learning algorithm.
When I concatenate them,
should I scale and normalize them separately first and then concatenate?
Or should I concatenate them first and scale-normalize them together?
This is too broad question and must be answered carefully. What are your features? If you have categorical features, you are better to encode it in one-hot manner and then you may circumvent scaling problems. Also, it depends on a classifier you want to use downstream : SVM will not tolerate badly scaled and uncentered data, but XGBoost will do ( to some degree )
Without much details it is hard to know whether rescaling your data will improve the performance of your algorithms before you apply them. If often can, but not always.
A good tip is to create rescaled copies of your dataset and race them against each other using your test harness and a handful of algorithms you want to spot check. This can quickly highlight the benefits (or lack there of) of rescaling your data with given models, and which rescaling method may be worthy of further investigation.
As mentioned above you could train two separate classifiers on both the data. This is useful because say you have more faith in the CNN features than histogram based features then you give that classifier's prediction more weight in the final output. Once you concatenate the features you are unable to use make use of this prior knowledge. If both features sets are equally informative with similar noise then concatenation should work fine as well.
