Is it necessary to normalize (Z-score) the dataset (high dimension) when the dimensionality of features varies greatly?

If I normalize the dataset, then the probability density (f1) obtained by KDE using the normalized dataset should not be equal to the probability density (f2) obtained by KDE using the dataset directly, so how to convert f1 to f2 after getting f1?

  • $\begingroup$ I think that by "dimensionality of features" you meant scaling of the features? Z-scaling does not do anything about dimensionality, e.g. if you are dealing with a matrix of N samples and K variables, than Z-scaling would result with you having N*K matrix as output, so the dimensionality (K) does not change. $\endgroup$
    – Tim
    Aug 3 '20 at 9:21
  • $\begingroup$ sorry, I didn't make it clear. Not doing dimensionality reduction, but data normalization. $\endgroup$
    – Gid
    Aug 3 '20 at 9:56

The only problem with multivariate kernel density estimation could be if you assumed that for all the variates you use same bandwidth, but this is not what people usually do, at least unless they have good reason for doing so. Usually we either use different bandwidth parameters per each variate, or scale the empirical covariance matrix, to pick the bandwidth. In both, latter cases, it is not a problem that variates have different scales, since we use different bandwidths for them.

  • $\begingroup$ Thanks Tim. I use 'Scotts_factor * np.sqrt(dataset_covariance_matrix)' for bandwidth, does that mean I'm using the 'scale the empirical covariance matrix' method? If so, can I use datasets directly without normalization ? $\endgroup$
    – Gid
    Aug 3 '20 at 9:48
  • $\begingroup$ @Gid from what you've said, you seem to be scaling based on covariance matrix, so there is no need of z-scaling the data. $\endgroup$
    – Tim
    Aug 3 '20 at 10:02
  • $\begingroup$ Got it, thx Tim $\endgroup$
    – Gid
    Aug 3 '20 at 10:40
  • $\begingroup$ @Gid: If this post is useful you may consider upcoming it. (And if it answers the issue reported, accepting it as an answer.) Regarding the statistics aspect of the question: Higher dimensional densities are notoriously difficult to compute. You do not mention how many dimensions you are dealing with but consider a dimensionality reduction step before density estimation if the data dimensions are in double figures. $\endgroup$
    – usεr11852
    Aug 3 '20 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.