I am trying to identify relevant features for a problem. The features are discrete and continuous in [0,1]. The target variable is [0,1]. I have tried linear regression by standardizing(subtract mean and divide by standard deviation) all the features. For example, 10,4,0.5,0.3 and the target is 0.6. I have two doubts: i) do I need to standardize the data for linear regression (I am getting p-value of 0 for most of the features) ii) Should I use beta regression, mixture model or mutual information regression. Do these methods need standardization.


The best method for your problem is Logistic Regression, also called Logit Regression. This is a variation of OLS Regression that specifically caters to a binomial dependent variable (0,1). Logistic Regression is very flexbible. While the Y variable has to be (0, 1), the Xs variables can be pretty much anything you want: binomial (0, 1), discretionary, continuous.

When using Logistic Regression, I don't think you will get much benefit from standardizing variables. That's because it just does not make that much sense when using binomial variables (0, 1).

If you care to standardize variables in your model, I would only standardize the continuous ones, and not the binomial ones.

  • $\begingroup$ Thanks for the reply. I also want to test which variable or feature is important. In linear regression, I can look at regression co-efficient and p-values. Can we do similar analysis for logistic regression? $\endgroup$
    – LCP
    Mar 29 at 16:20
  • $\begingroup$ Yes, exactly. Logistic Regression is really just another OLS Regression regressing the log of the odd transformation of a variable. The log of the odd is called an ogit. And, that's why it is also called Logit Regression. You will get regression coefficients, t-stats, p-values as regression output just like for OLS regression. $\endgroup$
    – Sympa
    Mar 29 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.