2
$\begingroup$

in psychological studies, you measure data from different participants. In order to be able to compare the responses between participants you standardize the data. The other day I was talking to a friend of mine which is a data scientist/compsci and they trained a SGDClassifier using scikit-learn without standardizing the data and a 10-fold cross-validation(cv). Afterwards she compared the errors of each cv. My question now, is the comparison meaningful if the data was not standardized? With each k-fold I would assume that you might get different means and stds and the comparison is off?

Can anyone enlighten on this topic?

$\endgroup$
3
  • $\begingroup$ Hey, @RSale can give us the information of what you treat as samples and as variables?? If I understand well, you have a bunch of participants for which you measure a bunch of variables ?? $\endgroup$
    – Fiodor1234
    Commented Feb 26, 2021 at 12:34
  • $\begingroup$ @Fiodor1234 It was really more of a general question. I didn't have a particular data set in mind. Shouldn't you just in general standardize the data? $\endgroup$
    – RSale
    Commented Feb 26, 2021 at 15:24
  • 1
    $\begingroup$ In general, you standardize your data for making more robust calculations. Imagine a variable that takes values from $[1000-2000]$ and a variable that takes values from $[0-1]$. The weight of the former one will dominate the weight of the second one, so that is one reason for standardizing your data. So, I believe what you are looking for is what makes your CV calculation more robust, instead of looking if the comparison of CV between participants is meaningful. $\endgroup$
    – Fiodor1234
    Commented Feb 26, 2021 at 15:56

1 Answer 1

1
$\begingroup$

Since classification problem you have the targets, and a particular type of logistic regression with SGD:

You may not standardize the data and still get the valid results. Also some data cannot be standardized (categorical for instance). For the numerical features you may standardize the data and this will help the classifier. Also you may bin the data, this may help as well, even for LR case, you never knows.

You may also try to scale the data to the [0,1] or into [-1,1] and see if this new feature helps. Different transformations on data may also be helpful, such as you may $log$ a feature to create a new one.

Further more you may do arithmetic operations on features to create new features, this may help.

The idea of K fold CV is to average the estimators at the end and get this as a valuable information on what parameter or model you selected works the best with your data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.