I am working on a basic house price prediction problem with traditional ML algorithms, not NN since the size of data is small comparing to the number of features.

The issue I am having is that many numerical features such as the size of lot area or size of basement become negative after standardization. Is it okay to use this way? Or do I need to do something about it?

Since I know there are outliers, I am not considering normalization. Cleaning some outliers would make the data even smaller.

  • $\begingroup$ What is your standardization method if it’s not the z-transform? If you did a z-transform, while it may look silly to talk about a lot size of -1, that just means a lot size one standard deviation below average. (Or by “not considering normalization” do you mean something other than the z-transform?) $\endgroup$
    – Dave
    Sep 28 '19 at 16:08
  • 1
    $\begingroup$ Best not to remove outliers unless they are errors $\endgroup$
    – mkt
    Sep 28 '19 at 16:17
  • $\begingroup$ @Dave Thanks for the comment. By standardization, I meant z-transform. I knew that this makes some data negative since we subtract mean value. I just wasn't sure if that's okay for features that only have non-negative values. By normalization, I meant min-max normalization which is affected a lot by outliers. $\endgroup$
    – lovemath
    Sep 29 '19 at 15:20
  • $\begingroup$ @mkt Thanks for the comment. $\endgroup$
    – lovemath
    Sep 29 '19 at 15:24

Every variable you standardize should end up having negative values. This wont be a problem for any ML algorithms. As long as you're just doing this for the predictors and not the response variable this shouldn't cause you any problems.

  • 2
    $\begingroup$ Thanks for the answer. I knew that standardization makes some data negative since we subtract mean value. However, I wasn't sure if it is okay for all positive features. You said this won't be a problem for any ML algorithms. I wonder if I can find something to back it up or is it more like heuristic? $\endgroup$
    – lovemath
    Sep 29 '19 at 15:12
  • $\begingroup$ What do you think the problem might be? The model will just be a function that maps values in your feature space to predicted house prices, I don't see any reason that function wouldn't be able to handle negative real numbers. Are you worried about the predicted house price being negative? $\endgroup$ Sep 29 '19 at 15:28
  • $\begingroup$ There are features which are originally a mixture of positive and negative values and features with only non-negative values. I wasn;t sure that, by turning a non-negative valued feature into a mixture feature, it could change some characteristics about the feature. Of course, model will run no matter what. $\endgroup$
    – lovemath
    Sep 29 '19 at 15:39
  • 1
    $\begingroup$ So if you're using linear regression, you'd just be changing the interpretation of the coefficient for those values by standardizing the variables, but you wouldn't affect the fitted values of model itself. If you're using either random forest or gbms, those are insensitive to monotonic transformations of the predictors, so standardizing wont accomplish anything. What algorithms are you considering? More info on linear regression, and scaling $\endgroup$ Sep 29 '19 at 15:56
  • $\begingroup$ Thank you for the comment! I am actually using many different models for experiments including lin. regs and tree-based models. $\endgroup$
    – lovemath
    Oct 2 '19 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.