8
$\begingroup$

Imagine you have many observations on which you want to run a classification algorithm. Each observation is characterized by a matrix of non-negative values. For all observations 90-98% of the values are 0.

To ensure that a machine learning algorithm perform the best, it is often recommended to do a feature standardization (see e.g. http://ufldl.stanford.edu/wiki/index.php/Data_Preprocessing). However, with a normal feature standardization the sparse cells obtain a value of roughly -0.25. I'm really interested in the benefit (both learning-wise and computationally) by letting all the cells with 0 stay 0 so the matrix continues to be sparse. One scaling method that achieves this is the 0-1 scaling. However, that is problematic if there's some extreme outliers in the data, which there are.

The data is modeled using a convolutional neural network, which I'm training with a stochastic gradient descent algorithm. What kind of feature scaling would make the most sense from an empirical and theoretical point of view?

$\endgroup$
2
  • 1
    $\begingroup$ I have a similar though less pronounced dataset where 25-50% of the values are zero. Did you find an answer or a technique? $\endgroup$
    – Anonymous
    Oct 21, 2017 at 20:27
  • 1
    $\begingroup$ I don't remember exactly. Do you have any outliers for each feature? Otherwise, I think simple 0-1 scaling works great. $\endgroup$
    – pir
    Oct 21, 2017 at 21:57

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.